Application classloaders leaked by log4j2 jmx beans

Environment

• JBoss Enterprise Application Platform (EAP) 7.x
• log4j2

Issue

• We reach a metaspace OOME after repeated application redeploys. A heap dump shows application classloaders persisted by jmx beans that are still regsitered from log4j2:

class java.lang.management.ManagementFactory @ 0x6c12a69b0 System Class
'- platformMBeanServer org.jboss.as.jmx.PluggableMBeanServerImpl @ 0x6c1547220
   '- rootMBeanServer org.jboss.as.jmx.PluggableMBeanServerImpl$TcclMBeanServer @ 0x6c1547248
      '- delegate com.sun.jmx.mbeanserver.JmxMBeanServer @ 0x6c1547260
         '- mbsInterceptor com.sun.jmx.interceptor.DefaultMBeanServerInterceptor @ 0x6c15473d8
            '- repository com.sun.jmx.mbeanserver.Repository @ 0x6c1547400
               '- domainTb java.util.HashMap @ 0x6c1547420
                  '- table java.util.HashMap$Node[32] @ 0x6c0b27310
                     '- [22] java.util.HashMap$Node @ 0x6c15482f8
                        '- next java.util.HashMap$Node @ 0x6c298e7c0
                           '- next java.util.HashMap$Node @ 0x6cbc3f718
                              '- value java.util.HashMap @ 0x6cbc3f738
                                 '- table java.util.HashMap$Node[1024] @ 0x6f89acdf8
                                    |- [800] java.util.HashMap$Node @ 0x719d8a4b0
                                    |  '- value com.sun.jmx.mbeanserver.NamedObject @ 0x719d8a568
                                    |     '- object com.sun.jmx.mbeanserver.StandardMBeanSupport @ 0x719d8a880
                                    |        '- perInterface com.sun.jmx.mbeanserver.PerInterface @ 0x7196649f8
                                    |           '- mbeanInterface class org.apache.logging.log4j.core.jmx.AppenderAdminMBean @ 0x719310be8
                                    |              '-  org.jboss.modules.ModuleClassLoader @ 0x7192d6560

Resolution

• If you do not require JMX mbeans, you could try disabling log4j2's JMX features by adding the following to your JVM options:

-Dlog4j2.disableJmx=true

Root Cause

• log4j2 is included in the application, but it is not cleaning up and unregistering its created jmx beans. Disabling with the flag: log4j2.disableJmx=true, following Content from logging.apache.org is not included.Log4j Manual

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.