10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

Getting Started New sletters Store

Welcome, Guest

Login

Register

Search the Community

Products Industries Lines of Business

Services & Support Training & Education University Alliances

About SCN Partnership Events & Webinars

Downloads Developer Center Innovation

Technology Troubleshooting Guide

(JSTTSG)(Deploy)Thread Dump
Added by Ina Ivanova, last edited by Tapan Goyal on Sep 24, 2013

Overview
You are using AS Java and w ant to create a thread dump from all Java threads running on a server process. This page describes how to perform a thread dump and how to locate the collected information. You have to create 3 consecutive thread dum ps at a short interval of time (e.g. 5 seconds).

How can I generate thread dumps ?

#Using the SAP Management Console #Using JCMon #On Unix #On Window s #On IBM i Without having console on Window s (Java 1.4 or 5)

Using the SAP Management Console

Step 1 Open SAP(M)MC.

wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump

1/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

Step 2 Go to AS Java Process Table view . Step 3 Right click on the server process for w hich you w ant to create a thread dump and select Dum p Stack Trace . If you are not sure which is the right server process, create thread dumps for all server processes . Step 4 Wait for at least 5 seconds. Step 5 Make another dump (until you have the 3 dumps). In case you want to make dumps for all server processes, you should have 3*<number of nodes> dumps .

Using JCMon
The JCMon must be started w ith adm user in order to have the correct environment (for example LD_LIBRARY_PATH). Step 1 Start the JCMon program w ith the profile of the instance. The JCMon binary is located into one of the follow ing directories: /usr/sap/<SID>/<Instance>/j2ee/os_libs /usr/sap/<SID>/SYS/exe/run directory. Example: >jcmon.exe pf=C:\usr\sap\N71\SYS\profile\N71_JC02_sofd60135707 Netw eaver Versions 6.40-7.0x Step 2 Enter "20" to go to the "Local Administration Menu".
wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump 2/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

Step 3 Choose the "Dump stacktrace" menu item by entering its number. Step 4 Select the server process from the list of processes above (the table just above the menu items) and enter the value from the "Idx" column. Step 5 Confirm the command w ith a "y" if the selected server process is correct. Netw eaver Versions 7.10 and higher versions: Step 2 Enter "30" to go to the "Solid Rock Administration Menu". Step 3 Choose the "Dump call stacks" menu item by entering 10 Step 4 Select the server process from the list of processes above (the table just above the menu items) and enter the value from the "Idx" column. Step 5 Confirm the command w ith a "y" if the selected server process is correct.

Using JSMon
Getting thread dump via JSMon tool is applicable to SAP NetWeaver 7.1 EHP1 and higher versions. Start the JSMon program w ith the profile of the instance. >jsmon pf=/usr/sap/<SID>/SYS/profile/<SID_Instance_NostName> The JSMon binary is located in the follow ing path: /usr/sap/<SID>/<Instance>/exe directory or in the: /usr/sap/<SID>/SYS/exe/uc/<OS_Name> directory. In the started prompt please follow the steps: Page 2 1) Get the index of the process w hich w ill create a full thread dump > process view 2) Create full thread dump (on Netw eaver 7.11) > process vmdump <idx> On Netw eaver 720 and higher: > process sdump <idx> After completion, you w ill find the thread dump in the
wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump 3/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

/usr/sap/<SID>/<Instance>/w ork/std_<node name>.out file The thread dump could be found in the file system of the application server under location /usr/sap/<SYS>/<INSTANCE>/w ork/dev_sever[0..n] or std_server[0..n].

Where are the generated thread dumps?

The location of the generated thread dump is JVM dependent: For JVMs from SUN, SAP and HP - /usr/sap/<SID>/<Instance>/work/std<node nam e>.out_' For JVMs from IBM /usr/sap/<SID>/<Instance>/j2ee/cluster/serverX/javacore.<timestamp>.<PID>.txt If you have problems locating the thread dump, it is a good idea to have a look at the various files in the w ork folder. After you triggered the thread dump, sort the files in the w ork folder by date. Then open the most recent ones and look for the w ords dum p and thread. In case the process is not AS Java but some SL tool like sl_auto, SAPInst or JSPM, please see this page .

Using NWA (NetWeaver Administrator)

Thread dumps can be captured through NWA for NW AS Java 7.1 and above systems Step 1 Open a brow ser and access NWA URL http://<AS Java Hostname>:<HTTP Port>/nw a Step 2 Navigate to Troubleshooting -> Advanced Troubleshooting -> Thread Dump Analysis

wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump

4/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

Step 3 Click Generate Thread Dump. Select an option that best suits to your situation. Press OK.

wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump

5/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

Step 4 A Thread Dump w ill be generated that can be dow nloaded and attached w ith CSS message for further analysis.

wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump

6/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

Alternative procedures
On Unix
Step 1 Start the JCMon program w ith the profile of the instance. Step 2 Enter "20" to go to the "Local Administration Menu". Step 3 Note the process ID (PID) from the server process. Step 4 Issue a "kill -3 <PID>" command from the shell. Step 5 Find the generated thread dump (see the section above).

On Windows
This procedure is not applicable for double stack installations (R/3 and AS Java running in the same instance). Step 1 Open the SAP Management Console. Step 2 Navigate to the Process table of the Java instance. Step 3 Right-click on the server process and from the context menu, choose "Dump Stack Trace".

On IBM i
Step 1 Log on as QSECOFR or a user w ith similar authorization. Step 2 To check the Java processes in your subsystem, enter command WRKACTJOB SBSR3(<instance_number>) . Jobs labeled "PGM-JLAUNCH" in column "Function" are Java processes. In earlier support packages, those processes had the job name "JLAUNCH", w ith higher SP levels, they have job names "SDM", "DISPATCHER", "SERVER0" etc. Substitute <name> in the follow ing lines accordingly. Step 3 Press function key F11 tw ice to get the user und the process number of the JVMs (job name <name>). These values are needed to create a Java thread dump. Step 4
wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump 7/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

To create the Java thread dump, enter command: DMPJVM JOB(<process_number>/<job user>/<name>) STACKFRAME(*ALL) DMPJVM creates a spool file. In this spool file you can find not only all Java threads dumps, but also the heap stack and the class loader information. DMPJVM does not stop a running JVM. This means that you can create a Java thread dump at any time w ithout stopping a JVM. Step 5 To look at the created spool file, enter command WRKSPLF. Use the function key F11 to collect the follow ing information: file number, job name and job number. Step 6 Create a text file containg the needed Java thread dump information to make the information accessible also to users of other operation systems. Perform the next steps on your Window s PC.

Unattach the file getJavaDumpFile.TXT from SAPNote 710154 into a temporary directory on your Window s PC. Rename the file to getJavaDumpFile.bat. Run the batch file to get a file called javadump.txt containing the Java thread dumps: getJavaDumpFile.bat <iSeries_hostname> <QSECOFR> <QSECOFR_password> <spoolfile_name> <file_number> <job_name> <job_number> Example: _getJavaDumpFile.bat as0096 QSECOFR secret QDMPJVM 2 QPADEV0004 QSECOFR 060594_

Using jstack
If you are running Sun/SAP Java EE 6 (Window s and Linux) or Java EE 5 (Linux only) you may use the jps and jstack tools to obtain a thread dump. Executing jps gives a list of all the Java processes on the current host, along w ith their process IDs (PID). Executing jstack and providing a PID as the only argument yields a full thread dump of the respective process:

C : \ u s r \ s a p \ L K G \ J 9 2 \ e x e \ s a p j v m _ 6 \ b i n > j p s 8 8 4 4M a i n 1 0 2 2 0J p s C : \ u s r \ s a p \ L K G \ J 9 2 \ e x e \ s a p j v m _ 6 \ b i n > j s t a c k8 8 4 4 2 0 0 9 0 1 3 01 0 : 2 6 : 4 0 F u l lt h r e a dd u m pS A PJ a v aS e r v e rV M( 6 . 1 . 0 8 5 1D e c1 62 0 0 80 5 : 1 9 : 2 9-c o n s-o p t U-w i n d o w sx 8 6-6-b a s 2 : 1 1 3 1 8 5( m i x e d m o d e ) ) : " L o wM e m o r yD e t e c t o r "d a e m o nc p u = 0 . 0 0[ r e s e t0 . 0 0 ]m sa l l o c a t e d = 1 6 2 7 2B( 1 5 . 8 9k B )[ r e s e t1 6 2 7 2B( 1 5 . 8 9k B ) ] i o =f i l ei / o :0 / 0B ,n e ti / o :0 / 0B ,f i l e so p e n e d : 0 ,s o c k so p e n e d : 0 [ r e s e tf i l ei / o :0 / 0B ,n e ti / o :0 / 0B ,f i l e s o p e n e d : 0 ,s o c k so p e n e d : 0] p r i o = 6t i d = 0 x 0 0 a 4 b 2 9 8 [ _ t h r e a d _ b l o c k e d( _ a t _ s a f e p o i n t ) ,s t a c k ( 0 x 0 0 e 6 0 0 0 0 , 0 x 0 0 e b 0 0 0 0 ) ][ 0 x 0 0 0 0 0 0 0 0 . . 0 x 0 0 0 0 0 0 0 0 ]
wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump 8/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

j a v a . l a n g . T h r e a d . S t a t e :R U N N A B L E . . . . . . . . . . . . . . . p r i o = 6t i d = 0 x 0 0 6 2 4 4 0 0 [ _ t h r e a d _ b l o c k e d( _ a t _ s a f e p o i n t ) ,s t a c k ( 0 x 0 0 4 2 0 0 0 0 , 0 x 0 0 4 7 0 0 0 0 ) ][ 0 x 0 0 4 6 f 0 0 0 . . 0 x 0 0 4 6 f e 0 4 ] j a v a . l a n g . T h r e a d . S t a t e :T I M E D _ W A I T I N G( s l e e p i n g ) a tj a v a . l a n g . T h r e a d . s l e e p ( J ) V ( N a t i v eM e t h o d ) a tM a i n . m a i n ( [ L j a v a / l a n g / S t r i n g ; ) V ( M a i n . j a v a : 1 3 ) . . . . . . . . . . . . . . . . " V MP e r i o d i cT a s kT h r e a d "c p u = 0 . 0 0[ r e s e t0 . 0 0 ]m s i o =f i l ei / o :0 / 0B ,n e ti / o :0 / 0B ,f i l e so p e n e d : 0 ,s o c k so p e n e d : 0 [ r e s e tf i l ei / o :0 / 0B ,n e ti / o :0 / 0B ,f i l e s o p e n e d : 0 ,s o c k so p e n e d : 0] p r i o = 1 0t i d = 0 x 0 0 a 4 c a a 8 J N Ig l o b a lr e f e r e n c e s :1 1 0 5 C : \ u s r \ s a p \ L K G \ J 9 2 \ e x e \ s a p j v m _ 6 \ b i n >

Without having console on Windows (Java 1.4 or 5)

1. Go to http://w w w .adaptj.com/main/dow nload 2. Clink "Launch" -> w ill see...

wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump

9/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

3. Process (drop dow n menu) -> Select -> <choose java process> OK (button) 4. Process (drop dow n menu) -> Thread Dump 5. Done NOTE: This doesn't alw ays w ork. If you are connected to the target system via term inal connection (i.e. m stsc - rem ote desktop) you m ight need to perform som e additional steps. For m ore info see the tool docum entation. If possilbe alw ays use the standard tools that com e w ith the JVM (jps and jstack)

How to read Thread Dump file(s)?

Open the thread dump file (read "Where are the generated thread dumps?" section if you are not able to locate the thread dump file).
wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump 10/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

For the given sample Notepad.exe is used.

1. Go to the bottom of the file the and perform search for "Full thread dump SAP Java Server"

Note The search is performed from BOTTOM to TOP of the file in order to locate the last thread dump

2. Put the cursor in the beginning of the thread dump

3. Scroll the file until you reach the end of thread dump

wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump

11/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

4. Copy/paste the thread dump information into new file 5. Perform a search in the new file that contains only the thread dump for keywords you are looking for.
E.g. to locate and analyze the Deployment threads look for: DeployThread com.sap.engine.services.dc com.sap.engine.services.deploy

6. For two or more thread dump files compare the called classes and at which row the class is called in order to determine if the process is still running or is hanging.

wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump

12/13

10/18/13

(JSTTSG)(Deploy)Thread Dump - Technology Troubleshooting Guide - SCN Wiki

No labels
Contact Us Privacy SAP Help Portal Terms of Use

Legal Disclosure

Copyright

Follow SCN

wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(Deploy)Thread+Dump

13/13