Getting "Unable to open socket file" message when executing jstack / jmap / jcmd and unable to generate a thread dump / heap dump

Environment

• Red Hat JBoss Enterprise Application Platform (EAP)
• JDK

Issue

• Got the following "Unable to open socket file: target process not responding or HotSpot VM not loaded" when executing jstack command and unable to generate a thread dump:

  [root@myhost /tmp]# jstack -l <JAVA_PID>
  <JAVA_PID>: Unable to open socket file: target process not responding or HotSpot VM not loaded 
  The -F option can be used when the target process is not responding
  

• Got the following "Unable to open socket file: target process not responding or HotSpot VM not loaded" when executing jmap command and unable to generate a heap dump:

  [root@myhost /tmp]# jmap -dump:format=b,file=heap.hprof <JAVA_PID>
      <JAVA_PID>: Unable to open socket file: target process not responding or HotSpot VM not loaded 
      The -F option can be used when the target process is not responding
  

• Same error was thrown when executing jcmd <JAVA_PID> GC.heap_dump -all=true FILENAME.hprof

• Got the above message when executing jmap -histo:live <PID> command.

Resolution

• Make sure to execute jstack and jmap command from same uid/gid to the java process.

• Execute jstack or jmap command as JBoss/Java user. For example:

  1. Check PID and user of Java process by executing ps command:

      ps aux | grep java
     

  2. Execute jstack command as JBoss user:

      $su - <JAVA_USER>
      $jstack -l <JAVA_PID>
     

     or

      #sudo -u <JAVA_USER> jstack -l <JAVA_PID>
     

     or

      #su - <JAVA_USER> -c "jstack -l <JAVA_PID>"
     

• For thread dump, you can use "kill -SIGQUIT <JAVA_PID>" or "kill -3 <JAVA_PID>" instead. This will output thread dump to console output.

• Another possible reason for this issue is that the /tmp directory is not writable by your JVM user. /tmp is used to store hsperfdata utilized by jstack and jmap.

  3. Execute jstack / jmap with the '-F' option. For example: jmap -F -dump:format=b,file=heap.hprof JBOSS_PID.

• Execute jcmd with an absolute path for the heap dump file. For example: jcmd <JAVA_PID<> GC.heap_dump -all=true /jboss/heapdumps/heap.hprof

Root Cause

The following are known causes:

• jstack or jmap command is not executed from JBoss/Java user but it is executed from another user (like root).
• /tmp/ directory is not writeable by the JBoss user.
• The JVM is under stress or doing many active garbage collections, and the -F option is required.
• The '/tmp/.java_pid<pid>' file is missing. This is a socket file that jmap/jstack/jcmd use to communicate with VM.

Diagnostic Steps

Check if the '/tmp/.java_pid<pid>' file is missing.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.