前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >Jvm故障处理工具

Jvm故障处理工具

作者头像
你的益达
发布2020-08-27 15:07:14
1K0
发布2020-08-27 15:07:14
举报
文章被收录于专栏:阿伟的个人博客
jps

类似linux中的ps命令,jps(JVM Process Status),Java虚拟机进程状态。

代码语言:javascript
复制
jps [option] [hostid]

option : 
    -m 输出虚拟机启动时传给main()的参数
    -l 输出主类全限定类名
    -v 输出虚拟机进程启动时JVM的参数
代码语言:javascript
复制
[root@gww ~]# jps -m
26338 FullGCProblem
26597 Jps -m
[root@gww ~]# jps -l
26338 FullGCProblem
26652 sun.tools.jps.Jps
[root@gww ~]# jps -v
26338 FullGCProblem -Xms200M -Xmx200M -XX:+PrintGC
26677 Jps -Dapplication.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64 -Xms8m
jinfo:Java配置信息

configuration Info for java,实时查看和调整虚拟机的各项参数。

代码语言:javascript
复制
jinfo [option] pid
代码语言:javascript
复制
[root@gww ~]# jinfo 27161
Attaching to process ID 27161, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.242-b08
Java System Properties:

java.runtime.name = OpenJDK Runtime Environment
java.vm.version = 25.242-b08
sun.boot.library.path = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/amd64
java.vendor.url = http://java.oracle.com/
java.vm.vendor = Oracle Corporation
path.separator = :
file.encoding.pkg = sun.io
java.vm.name = OpenJDK 64-Bit Server VM
sun.os.patch.level = unknown
sun.java.launcher = SUN_STANDARD
user.country = US
user.dir = /home/gw/java
java.vm.specification.name = Java Virtual Machine Specification
java.runtime.version = 1.8.0_242-b08
java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
os.arch = amd64
java.endorsed.dirs = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/endorsed
java.io.tmpdir = /tmp
line.separator = 

java.vm.specification.vendor = Oracle Corporation
os.name = Linux
sun.jnu.encoding = UTF-8
java.library.path = /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
java.specification.name = Java Platform API Specification
java.class.version = 52.0
sun.management.compiler = HotSpot 64-Bit Tiered Compilers
os.version = 3.10.0-1062.18.1.el7.x86_64
user.home = /root
user.timezone = 
java.awt.printerjob = sun.print.PSPrinterJob
file.encoding = UTF-8
java.specification.version = 1.8
user.name = root
java.class.path = .
java.vm.specification.version = 1.8
sun.arch.data.model = 64
sun.java.command = FullGCProblem
java.home = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre
user.language = en
java.specification.vendor = Oracle Corporation
awt.toolkit = sun.awt.X11.XToolkit
java.vm.info = mixed mode
java.version = 1.8.0_242
java.ext.dirs = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/ext:/usr/java/packages/lib/ext
sun.boot.class.path = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/jfr.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/classes
java.vendor = Oracle Corporation
file.separator = /
java.vendor.url.bug = http://bugreport.sun.com/bugreport/
sun.io.unicode.encoding = UnicodeLittle
sun.cpu.endian = little
sun.cpu.isalist = 

VM Flags:
Non-default VM flags: -XX:CICompilerCount=2 -XX:InitialHeapSize=209715200 -XX:MaxHeapSize=209715200 -XX:MaxNewSize=69861376 -XX:MinHeapDeltaBytes=196608 -XX:NewSize=69861376 -XX:OldSize=139853824 -XX:+PrintGC -XX:+UseCompressedClassPointers -XX:+UseCompressedOops 
Command line:  -Xms200M -Xmx200M -XX:+PrintGC
jstack

java堆栈跟踪工具,用于生成虚拟机当前时刻的线程快照。

该命令可以用于查死锁。此外还可以查询CPU占用率。

代码语言:javascript
复制
[root@gww ~]# jstack 27161 | more
2020-08-25 14:10:39
Full thread dump OpenJDK 64-Bit Server VM (25.242-b08 mixed mode):

"Attach Listener" #58 daemon prio=9 os_prio=0 tid=0x00007f86ac002000 nid=0x6aff waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"pool-1-thread-50" #57 prio=5 os_prio=0 tid=0x00007f86d41a7000 nid=0x6a59 waiting on condition [0x00007f86b03a4000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000f6d722d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"pool-1-thread-49" #56 prio=5 os_prio=0 tid=0x00007f86d41a5000 nid=0x6a58 waiting on condition [0x00007f86b04a5000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000f6d722d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
jmap
arthas

下载及使用

代码语言:javascript
复制
curl -O https://arthas.aliyun.com/arthas-boot.jar
java -jar arthas-boot.jar

进入之后会出现多个java进程,此时输入需要attach的arthas编号,即可进入arthas终端。

代码语言:javascript
复制
[root@gww ~]# java -jar arthas-boot.jar
[INFO] arthas-boot version: 3.3.9
[INFO] Found existing java process, please choose one and input the serial number of the process, eg : 1. Then hit ENTER.
* [1]: 27922 FullGCProblem
1
[INFO] arthas home: /root/.arthas/lib/3.3.9/arthas
[INFO] Try to attach process 27922
[INFO] Attach process 27922 success.
[INFO] arthas-client connect 127.0.0.1 3658
  ,---.  ,------. ,--------.,--.  ,--.  ,---.   ,---.                           
 /  O  \ |  .--. ''--.  .--'|  '--'  | /  O  \ '   .-'                          
|  .-.  ||  '--'.'   |  |   |  .--.  ||  .-.  |`.  `-.                          
|  | |  ||  |\  \    |  |   |  |  |  ||  | |  |.-'    |                         
`--' `--'`--' '--'   `--'   `--'  `--'`--' `--'`-----'                          


wiki      https://arthas.aliyun.com/doc                                         
tutorials https://arthas.aliyun.com/doc/arthas-tutorials.html                   
version   3.3.9                                                                 
pid       27922                                                                 
time      2020-08-25 14:20:54        
dashboard命令

dashboard 仪表盘。

代码语言:javascript
复制
ID           NAME                                    GROUP                      PRIORITY     STATE        %CPU         TIME          INTERRUPTED  DAEMON       
54           pool-1-thread-47                        main                       5            TIMED_WAITIN 33           0:1           false        false        
47           pool-1-thread-40                        main                       5            WAITING      4            0:0           false        false        
8            pool-1-thread-1                         main                       5            WAITING      2            0:1           false        false        
56           pool-1-thread-49                        main                       5            WAITING      2            0:1           false        false        
17           pool-1-thread-10                        main                       5            WAITING      1            0:1           false        false        
18           pool-1-thread-11                        main                       5            WAITING      1            0:1           false        false        
19           pool-1-thread-12                        main                       5            WAITING      1            0:0           false        false        
20           pool-1-thread-13                        main                       5            WAITING      1            0:1           false        false        
21           pool-1-thread-14                        main                       5            WAITING      1            0:1           false        false        
22           pool-1-thread-15                        main                       5            WAITING      1            0:0           false        false        
23           pool-1-thread-16                        main                       5            WAITING      1            0:0           false        false        
24           pool-1-thread-17                        main                       5            WAITING      1            0:0           false        false        
25           pool-1-thread-18                        main                       5            WAITING      1            0:0           false        false        
26           pool-1-thread-19                        main                       5            WAITING      1            0:0           false        false        
Memory                            used        total      max         usage      GC                                                                             
heap                              123M        193M       193M        63.63%     gc.copy.count                           8                                      
eden_space                        39M         53M        53M         73.36%     gc.copy.time(ms)                        460                                    
survivor_space                    6M          6M         6M          100.00%    gc.marksweepcompact.count               0                                      
tenured_gen                       77M         133M       133M        57.93%     gc.marksweepcompact.time(ms)            0                                      
nonheap                           23M         24M        -1          97.07%                                                                                    
code_cache                        5M          5M         240M        2.29%                                                                                     
Runtime                                                                                                                                                        
os.name                                                                         Linux                                                                          
os.version                                                                      3.10.0-1062.18.1.el7.x86_64                                                    
java.version                                                                    1.8.0_242                                                                      
java.home                                                                       /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre               
systemload.average                                                              0.65                                                                           
processors                                                                      1                                                                              
uptime                                                                          462s            

定位cpu占比较高问题。直接使用arthas定位出来cpu占比较高的线程,此时分辨该线程属于业务线程还是系统线程。

若是业务线程,可查询代码中是否有出现死循环之类的问题

若是系统线程,则判断是否为gc线程

jvm命令

类似jinfo

代码语言:javascript
复制
[arthas@27922]$ jvm
 RUNTIME                                                                                                                                             
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 MACHINE-NAME                                 27922@gww                                                                                                        
 JVM-START-TIME                               2020-08-25 14:20:16                                                                                              
 MANAGEMENT-SPEC-VERSION                      1.2                                                                                                              
 SPEC-NAME                                    Java Virtual Machine Specification                                                                               
 SPEC-VENDOR                                  Oracle Corporation                                                                                               
 SPEC-VERSION                                 1.8                                                                                                              
 VM-NAME                                      OpenJDK 64-Bit Server VM                                                                                         
 VM-VENDOR                                    Oracle Corporation                                                                                               
 VM-VERSION                                   25.242-b08                                                                                                       
 INPUT-ARGUMENTS                              -Xms200M                                                                                                         
                                              -Xmx200M                                                                                                         
                                              -XX:+PrintGC                                                                                      
 CLASS-PATH                                   .                                                                                              
 BOOT-CLASS-PATH                              /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.8.0-openj 
                                              dk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/ 
                                              lib/sunrsasign.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/ja 
                                              va-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el 
                                              7_7.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/jfr.jar:/us 
                                              r/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/classes                                            
 LIBRARY-PATH                                 /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib                                                     

---------------------------------------------------------------------------------------------------------------------------------------------------------------
 CLASS-LOADING                                                                                                                                                 
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 LOADED-CLASS-COUNT                           2887                                                                                                             
 TOTAL-LOADED-CLASS-COUNT                     2897                                                                                                             
 UNLOADED-CLASS-COUNT                         10                                                                                                               
 IS-VERBOSE                                   false                                                                                             
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 COMPILATION                                                                                                                                                   
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 NAME                                         HotSpot 64-Bit Tiered Compilers                                                                                  
 TOTAL-COMPILE-TIME                           5217                                                                                                             
 [time (ms)]                                                                                                                                   
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 GARBAGE-COLLECTORS                                                                                                                                            
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 Copy                                         name : Copy                                                                                                      
 [count/time (ms)]                            collectionCount : 13                                                                                             
                                              collectionTime : 793                                                                                             

 MarkSweepCompact                             name : MarkSweepCompact                                                                                          
 [count/time (ms)]                            collectionCount : 5                                                                                              
                                              collectionTime : 3729                                                                                
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 MEMORY-MANAGERS                                                                                                                                               
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 CodeCacheManager                             Code Cache                                                                                                       

 Metaspace Manager                            Metaspace                                                                                                        
                                              Compressed Class Space                                                                                           

 Copy                                         Eden Space                                                                                                       
                                              Survivor Space                                                                                                   

 MarkSweepCompact                             Eden Space                                                                                                       
                                              Survivor Space                                                                                                   
                                              Tenured Gen                                                                                               
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 MEMORY                                                                                                                                                        
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 HEAP-MEMORY-USAGE                            init : 209715200(200.0 MiB)                                                                                      
 [memory in bytes]                            used : 188526928(179.8 MiB)                                                                                      
                                              committed : 202768384(193.4 MiB)                                                                                 
                                              max : 202768384(193.4 MiB)                                                                                       

 NO-HEAP-MEMORY-USAGE                         init : 2555904(2.4 MiB)                                                                                          
 [memory in bytes]                            used : 25741504(24.5 MiB)                                                                                        
                                              committed : 26607616(25.4 MiB)                                                                                   
                                              max : -1(-1 B)                                                                                              
 PENDING-FINALIZE-COUNT                       0                                                                                                        
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 OPERATING-SYSTEM                                                                                                                                              
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 OS                                           Linux                                                                                                            
 ARCH                                         amd64                                                                                                            
 PROCESSORS-COUNT                             1                                                                                                                
 LOAD-AVERAGE                                 2.4                                                                                                              
 VERSION                                      3.10.0-1062.18.1.el7.x86_64                                                                          
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 THREAD                                                                                                                                                        
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 COUNT                                        63                                                                                                               
 DAEMON-COUNT                                 12                                                                                                               
 PEAK-COUNT                                   64                                                                                                               
 STARTED-COUNT                                66                                                                                                               
 DEADLOCK-COUNT                               0                                                                                                 
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 FILE-DESCRIPTOR                                                                                                                                               
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 MAX-FILE-DESCRIPTOR-COUNT                    65535                                                                                                            
 OPEN-FILE-DESCRIPTOR-COUNT                   41        
thread命令

类似jstack

代码语言:javascript
复制
 thread       Display thread info, thread stack
 thread [-h] [-b] [id]
  OPTIONS:                                                                                                                                                      
 -h, --help                                           this help                                                                                                
 -b, --include-blocking-thread                        Find the thread who is holding a lock that blocks the most number of threads.                            
     --lockedMonitors                                 Find the thread info with lockedMonitors flag, default value is false.                                   
     --lockedSynchronizers                            Find the thread info with lockedSynchronizers flag, default value is false.                              
 -i, --sample-interval <value>                        Specify the sampling interval (in ms) when calculating cpu usage.                                        
     --state <value>                                  Display the thead filter by the state. NEW, RUNNABLE, TIMED_WAITING, WAITING, BLOCKED, TERMINATED is opt 
                                                      ional.                                                                                                   
 -n, --top-n-threads <value>                          The number of thread(s) to show, ordered by cpu utilization, -1 to show all.                             
 <id>                                                 Show thread stack    

使用thread -b即可查找死锁。

例如如下代码产生的死锁:

代码语言:javascript
复制
public class DeadLock {
    static Object o1 = new Object();
    static Object o2 = new Object();
    public static void main(String[] args) {
        new Thread(new Runnable() {
            @Override
            public void run() {
                synchronized (o1) {
                    try {
                        Thread.sleep(1);
                    } catch (InterruptedException e) {
                         e.printStackTrace();
                    }
                    synchronized (o2) { 
                    }
                }

            }
        }, "线程1").start();
        new Thread(new Runnable() {
            public void run() {
                synchronized (o2) {
                    try {
                        Thread.sleep(1);
                    } catch (InterruptedException e) {
                         e.printStackTrace();
                    }
                    synchronized (o1) { 

                    }
                }

            }
        },"线程2").start();
    }
}

使用thread -b检测结果如下:

代码语言:javascript
复制
[arthas@3454]$ thread -b
"线程1" Id=8 BLOCKED on java.lang.Object@74929bad owned by "线程2" Id=9
    at DeadLock$1.run(DeadLock.java:29)
    -  blocked on java.lang.Object@74929bad
    -  locked java.lang.Object@5df69221 <---- but blocks 1 other threads!
    at java.lang.Thread.run(Thread.java:748)

但是使用信号量写的死锁却无法检测出来

jad命令

反编译class文件。

heapdump命令

将堆导出为一个文件。然后可以使用图形界面软件查看.

只能用于测试机,不能用于生产环境。

生产环境中使用-XX:+HeapDumpOnOutOfMemoryError

其再oom时挂掉之前导出一份dump。

redefine命令

可以不用停止线上服务器进行修改线上代码的问题。

修改源码并重新编译,然后再arthas中使用

代码语言:javascript
复制
redefine /path/className.class

即可完成代码的修改。

document.querySelectorAll('.github-emoji') .forEach(el => { if (!el.dataset.src) { return; } const img = document.createElement('img'); img.style = 'display:none !important;'; img.src = el.dataset.src; img.addEventListener('error', () => { img.remove(); el.style.color = 'inherit'; el.style.backgroundImage = 'none'; el.style.background = 'none'; }); img.addEventListener('load', () => { img.remove(); }); document.body.appendChild(img); });

本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
原始发表:2020-08-25,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • jps
  • jinfo:Java配置信息
  • jstack
  • jmap
  • arthas
    • dashboard命令
      • jvm命令
        • thread命令
          • jad命令
            • heapdump命令
              • redefine命令
              相关产品与服务
              云服务器
              云服务器(Cloud Virtual Machine,CVM)提供安全可靠的弹性计算服务。 您可以实时扩展或缩减计算资源,适应变化的业务需求,并只需按实际使用的资源计费。使用 CVM 可以极大降低您的软硬件采购成本,简化 IT 运维工作。
              领券
              问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档