jcmd/jstack/jmap fail with AttachNotSupportedException

Environment

• OpenJDK
  • 8
  • 11
  • 17

Issue

• Executing the following command results in the exceptions below:

jcmd <pid> VM.native_memory

• OpenJDK8:

com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file: target process not responding or HotSpot VM not loaded
        at sun.tools.attach.LinuxVirtualMachine.<init>(LinuxVirtualMachine.java:106)
        at sun.tools.attach.LinuxAttachProvider.attachVirtualMachine(LinuxAttachProvider.java:63)
        at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:208)
        at sun.tools.jcmd.JCmd.executeCommandForPid(JCmd.java:147)
        at sun.tools.jcmd.JCmd.main(JCmd.java:131)

• OpenJDK11+:

com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file /proc/<pid>/root/tmp/.java_pid<pid>: target process <pid> doesn't respond within 10500ms or HotSpot VM not loaded

Resolution

Run jcmd/jstack/jmap on the JVM pid.

Run jcmd/jstack/jmap as the user of the Java process.

Run sudo jcmd/jstack/jmap to run with the elevated capabilities being applied to the user.

Use -XX:+StartAttachListener to create /proc/<pid>/root/tmp/.java_pid<pid> on JVM startup as a workaround for Content from bugs.openjdk.java.net is not included.JDK-8226919.

Disable SELinux. This has been known to prevent the socket connection.

Root Cause

The following are known causes:

• The pid is not a JVM.
• jcmd/jstack/jmap is not run as the same user and group as the java process.
• Content from bugs.openjdk.java.net is not included.JDK-8226919: attach in linux hangs due to permission denied accessing /proc/pid/root (OpenJDK11+)
• Elevated capabilities being applied to the user (e.g. sudo) or java executable (e.g. setcap cap_net_raw=+ep /usr/lib/jvm/<jre>/bin/java).

Diagnostic Steps

Use jps to verify the pid is a JVM and not some other type of process.

Verify jcmd/jstack/jmap is run as the same user as the java process.

Check <thegroup> of <theuser>:

su - <theuser> -c id
    uid=20104(<theuser>) gid=10078(<thegroup>)

Verify it matches <theuser> and <thegroup> of the /proc/<pid./root sym link to /:

# ls -al /proc/<pid>/ | grep root
lrwxrwxrwx.   1 <theuser> <thegroup> 0 Sep 28 09:31 root -> /

If the 'user:groupof/proc//rootdoes not match theuser:grouprunning the process, then there must be some elevated capabilities being applied to the user (e.g.sudo) or javaexecutable (e.g.setcap cap_net_raw=+ep /usr/lib/jvm//bin/java`).

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.