聊聊openjdk的jhsdb工具

March 27, 2019 · 5 min · jiezi

序本文主要研究一下openjdk的jhsdb工具sa-jdi.jarexport JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home"chmod +x $JAVA_HOME/lib/sa-jdi.jarjava -cp $JAVA_HOME/lib/sa-jdi.jar sun.jvm.hotspot.HSDBjava -cp $JAVA_HOME/lib/sa-jdi.jar sun.jvm.hotspot.CLHSDB在java9之前，JAVA_HOME/lib目录下有个sa-jdi.jar，可以通过如上命令启动HSDB(图形界面)及CLHSDB(命令行)sa-jdi.jar中的sa的全称为Serviceability Agent，它之前是sun公司提供的一个用于协助调试HotSpot的组件，而HSDB便是使用Serviceability Agent来实现的HSDB就是HotSpot Debugger的简称，由于Serviceability Agent在使用的时候会先attach进程，然后暂停进程进行snapshot，最后deattach进程(进程恢复运行)，所以在使用HSDB时要注意jhsdb/ # jhsdb clhsdb command line debugger debugd debug server hsdb ui debugger jstack –help to get more information jmap –help to get more information jinfo –help to get more information jsnap –help to get more informationjhsdb是java9引入的，可以在JAVA_HOME/bin目录下找到jhsdb；它取代了jdk9之前的JAVA_HOME/lib/sa-jdi.jarjhsdb有clhsdb、debugd、hsdb、jstack、jmap、jinfo、jsnap这些mode可以使用其中hsdb为ui debugger，就是jdk9之前的sun.jvm.hotspot.HSDB；而clhsdb即为jdk9之前的sun.jvm.hotspot.CLHSDBjhsdb jstack/ # jhsdb jstack –help –locks to print java.util.concurrent locks –mixed to print both java and native frames (mixed mode) –exe executable image name –core path to coredump –pid pid of process to attach–pid用于指定JVM的进程ID；–exe用于指定可执行文件；–core用于指定core dump文件异常jhsdb jstack –mixed –pid 1//……Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native Method) at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:534) at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:151)如果出现这个异常表示是采用jdk版本的问题，可以尝试一下其他jdk编译版本debugger/ # jhsdb jstack –locks –pid 1Attaching to process ID 1, please wait…Debugger attached successfully.Server compiler detected.JVM version is 12+33Deadlock Detection:No deadlocks found.“DestroyJavaVM” #32 prio=5 tid=0x000055c3b5be0800 nid=0x6 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE JavaThread state: _thread_blockedLocked ownable synchronizers: - None"http-nio-8080-Acceptor-0" #30 daemon prio=5 tid=0x000055c3b5d71800 nid=0x2f runnable [0x00007fa0d13de000] java.lang.Thread.State: RUNNABLE JavaThread state: _thread_in_native - sun.nio.ch.ServerSocketChannelImpl.accept0(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) @bci=0 (Interpreted frame) - sun.nio.ch.ServerSocketChannelImpl.accept(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) @bci=4, line=525 (Interpreted frame) - sun.nio.ch.ServerSocketChannelImpl.accept() @bci=41, line=277 (Interpreted frame) - org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept() @bci=4, line=448 (Interpreted frame) - org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept() @bci=1, line=70 (Interpreted frame) - org.apache.tomcat.util.net.Acceptor.run() @bci=98, line=95 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=835 (Interpreted frame)Locked ownable synchronizers: - <0x00000000e3aab6e0>, (a java/util/concurrent/locks/ReentrantLock$NonfairSync)“http-nio-8080-ClientPoller-0” #29 daemon prio=5 tid=0x000055c3b5c20000 nid=0x2e runnable [0x00007fa0d14df000] java.lang.Thread.State: RUNNABLE JavaThread state: _thread_in_native - sun.nio.ch.EPoll.wait(int, long, int, int) @bci=0 (Interpreted frame) - sun.nio.ch.EPollSelectorImpl.doSelect(java.util.function.Consumer, long) @bci=96, line=120 (Interpreted frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(java.util.function.Consumer, long) @bci=42, line=124 (Interpreted frame) - locked <0x00000000e392ece8> (a sun.nio.ch.EPollSelectorImpl) - locked <0x00000000e392ee38> (a sun.nio.ch.Util$2) - sun.nio.ch.SelectorImpl.select(long) @bci=31, line=136 (Interpreted frame) - org.apache.tomcat.util.net.NioEndpoint$Poller.run() @bci=55, line=743 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=835 (Interpreted frame)Locked ownable synchronizers: - None"http-nio-8080-exec-10" #28 daemon prio=5 tid=0x000055c3b48d6000 nid=0x2d waiting on condition [0x00007fa0d15e0000] java.lang.Thread.State: WAITING (parking) JavaThread state: _thread_blocked - jdk.internal.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame) - parking to wait for <0x00000000e3901670> (a java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=194 (Interpreted frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2081 (Interpreted frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=27, line=433 (Interpreted frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=107 (Interpreted frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=33 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=147, line=1054 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1114 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=628 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=835 (Interpreted frame) //……/ # jhsdb jstack –mixed –pid 1Attaching to process ID 1, please wait…Debugger attached successfully.Server compiler detected.JVM version is 12+33Deadlock Detection:No deadlocks found.—————– 47 —————–“http-nio-8080-Acceptor-0” #30 daemon prio=5 tid=0x000055c3b5d71800 nid=0x2f runnable [0x00007fa0d13de000] java.lang.Thread.State: RUNNABLE JavaThread state: _thread_in_native0x00007fa0ee0923ad ????????—————– 46 —————–“http-nio-8080-ClientPoller-0” #29 daemon prio=5 tid=0x000055c3b5c20000 nid=0x2e runnable [0x00007fa0d14df000] java.lang.Thread.State: RUNNABLE JavaThread state: _thread_in_native0x00007fa0ee05f3d0 epoll_pwait + 0x1d0x00007fa0daa97810 * sun.nio.ch.EPoll.wait(int, long, int, int) bci:0 (Interpreted frame)0x00007fa0daa91680 * sun.nio.ch.EPollSelectorImpl.doSelect(java.util.function.Consumer, long) bci:96 line:120 (Interpreted frame)0x00007fa0db85f57c * sun.nio.ch.SelectorImpl.lockAndDoSelect(java.util.function.Consumer, long) bci:42 line:124 (Compiled frame)* sun.nio.ch.SelectorImpl.select(long) bci:31 line:136 (Compiled frame)* org.apache.tomcat.util.net.NioEndpoint$Poller.run() bci:55 line:743 (Interpreted frame)0x00007fa0daa91c88 * java.lang.Thread.run() bci:11 line:835 (Interpreted frame)0x00007fa0daa88849 <StubRoutines>0x00007fa0ed122952 _ZN9JavaCalls11call_helperEP9JavaValueRK12methodHandleP17JavaCallArgumentsP6Thread + 0x3c20x00007fa0ed1208d0 _ZN9JavaCalls12call_virtualEP9JavaValue6HandleP5KlassP6SymbolS6_P6Thread + 0x2000x00007fa0ed1ccfc5 _ZL12thread_entryP10JavaThreadP6Thread + 0x750x00007fa0ed74f3a3 _ZN10JavaThread17thread_main_innerEv + 0x1030x00007fa0ed74c3f5 _ZN6Thread8call_runEv + 0x750x00007fa0ed4a477e _ZL19thread_native_entryP6Thread + 0xee//……–locks或者–mixed花费的时间可能比较长(几分钟，可能要将近6分钟)，因而进程暂停的时间也可能比较长，在使用这两个选项时要注意jhsdb jmapjmap -heap pid/ # jmap -heap 1Error: -heap option usedCannot connect to core dump or remote debug server. Use jhsdb jmap insteadjdk9及以上版本使用jmap -heap pid命令查看当前heap使用情况时，发现报错，提示需要使用jhsdb jmap来替代jhsdb jmap pid/ # jhsdb jmap 1sh: jhsdb: not found发现jlink的时候没有添加jdk.hotspot.agent这个module，添加了这个module之后可以发现JAVA_HOME/bin目录下就有了jhsdbPTRACE_ATTACH failed/ # jhsdb jmap 1You have to set –pid or –exe. <no option> to print same info as Solaris pmap –heap to print java heap summary –binaryheap to dump java heap in hprof binary format –dumpfile name of the dump file –histo to print histogram of java object heap –clstats to print class loader statistics –finalizerinfo to print information on objects awaiting finalization –exe executable image name –core path to coredump –pid pid of process to attach/ # jhsdb jmap –heap –pid 1Attaching to process ID 1, please wait…ERROR: ptrace(PTRACE_ATTACH, ..) failed for 1: Operation not permittedError attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can’t attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 1: Operation not permittedsun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException: Can’t attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 1: Operation not permitted at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:176) at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach(LinuxDebuggerLocal.java:336) at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.attachDebugger(HotSpotAgent.java:672) at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupDebuggerLinux(HotSpotAgent.java:612) at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:338) at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:305) at jdk.hotspot.agent/sun.jvm.hotspot.HotSpotAgent.attach(HotSpotAgent.java:141) at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:185) at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118) at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176) at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:326) at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:455)Caused by: sun.jvm.hotspot.debugger.DebuggerException: Can’t attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 1: Operation not permitted at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach0(Native Method) at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1AttachTask.doit(LinuxDebuggerLocal.java:326) at jdk.hotspot.agent/sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:151)发现PTRACE_ATTACH被docker禁用了，需要在运行容器时启用PTRACE_ATTACHdocker启用SYS_PTRACEdocker run –cap-add=SYS_PTRACE之后就可以正常使用jhsdb如下：/ # jhsdb jmap –heap –pid 1Attaching to process ID 1, please wait…Debugger attached successfully.Server compiler detected.JVM version is 12+33using thread-local object allocation.Shenandoah GC with 4 thread(s)Heap Configuration: MinHeapFreeRatio = 40 MaxHeapFreeRatio = 70 MaxHeapSize = 523763712 (499.5MB) NewSize = 1363144 (1.2999954223632812MB) MaxNewSize = 17592186044415 MB OldSize = 5452592 (5.1999969482421875MB) NewRatio = 2 SurvivorRatio = 8 MetaspaceSize = 21807104 (20.796875MB) CompressedClassSpaceSize = 1073741824 (1024.0MB) MaxMetaspaceSize = 17592186044415 MB ShenandoahRegionSize = 262144 (0.25MB)Heap Usage:Shenandoah Heap: regions = 1997 capacity = 523501568 (499.25MB) used = 70470552 (67.2059555053711MB) committed = 144441344 (137.75MB)jhsdb jinfo/ # jhsdb jinfo –help –flags to print VM flags –sysprops to print Java System properties <no option> to print both of the above –exe executable image name –core path to coredump –pid pid of process to attach使用jhsdb显示jinfo的sysprops如下：/ # jhsdb jinfo –sysprops –pid 1Attaching to process ID 1, please wait…Debugger attached successfully.Server compiler detected.JVM version is 12+33awt.toolkit = sun.awt.X11.XToolkitjava.specification.version = 12sun.jnu.encoding = UTF-8//……这个命令其实跟jinfo -sysprops 1是等价的jhsdb jsnap/ # jhsdb jsnap –pid 1Attaching to process ID 1, please wait…Debugger attached successfully.Server compiler detected.JVM version is 12+33java.threads.started=27 event(s)java.threads.live=24java.threads.livePeak=24java.threads.daemon=20java.cls.loadedClasses=8250 event(s)java.cls.unloadedClasses=1 event(s)java.cls.sharedLoadedClasses=0 event(s)java.cls.sharedUnloadedClasses=0 event(s)java.ci.totalTime=18236958158 tick(s)java.property.java.vm.specification.version=12java.property.java.vm.specification.name=Java Virtual Machine Specificationjava.property.java.vm.specification.vendor=Oracle Corporationjava.property.java.vm.version=12+33java.property.java.vm.name=OpenJDK 64-Bit Server VMjava.property.java.vm.vendor=Azul Systems, Inc.java.property.java.vm.info=mixed modejava.property.jdk.debug=release//……jhsdb jsnap的功能主要是由jdk.hotspot.agent模块中的sun.jvm.hotspot.tools.JSnap.java来提供的，它可以用于查看threads及class loading/unloading相关的event、JVM属性参数等，其中–all可以显示更多的JVM属性参数jhsdb与jcmdjhsdb: A New Tool for JDK 9这篇文章中列出了jhsdb与jcmd的等价命令，如下图：小结在java9之前，JAVA_HOME/lib目录下有个sa-jdi.jar，可以通过如上命令启动HSDB(图形界面)及CLHSDB(命令行)；sa-jdi.jar中的sa的全称为Serviceability Agent，它之前是sun公司提供的一个用于协助调试HotSpot的组件，而HSDB便是使用Serviceability Agent来实现的；HSDB就是HotSpot Debugger的简称，由于Serviceability Agent在使用的时候会先attach进程，然后暂停进程进行snapshot，最后deattach进程(进程恢复运行)，所以在使用HSDB时要注意jhsdb是java9引入的，可以在JAVA_HOME/bin目录下找到jhsdb；它取代了jdk9之前的JAVA_HOME/lib/sa-jdi.jar；jhsdb有clhsdb、debugd、hsdb、jstack、jmap、jinfo、jsnap这些mode可以使用；其中hsdb为ui debugger，就是jdk9之前的sun.jvm.hotspot.HSDB；而clhsdb即为jdk9之前的sun.jvm.hotspot.CLHSDBjhsdb在jdk.hotspot.agent这个模块中；对于jhsdb jstack的–locks或者–mixed命令花费的时间可能比较长(几分钟，可能要将近6分钟)，因而进程暂停的时间也可能比较长，在使用这两个选项时要注意；对于jdk9及以后的版本不再使用jmap -heap命令来查询heap内存情况，需要用jhsdb jmap –heap –pid来替代；使用jhsdb jmap需要在运行容器时启用PTRACE_ATTACH才可以docJVM信息查看jhsdbjdk.hotspot.agent jhsdbjhsdb: A New Tool for JDK 9jcmd: One JDK Command-Line Tool to Rule Them AllJVM in Docker and PTRACE_ATTACHServiceability in HotSpotThe HotSpot™ Serviceability Agent: An out-of-process high level debugger for a Java™ virtual machine