StackOverflow raised in EAP 6 by TRACE classloading logging

Solution Verified - Updated

Environment

  • JBoss Enterprise Application Platform (EAP) 6.x

Issue

  • We see a Stackoverflow like the following:
java.lang.StackOverflowError
	at org.jboss.modules.log.JDKModuleLogger.trace(JDKModuleLogger.java:129) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:203) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:459) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:408) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:389) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:134) [:1.3.3.Final-redhat-1]
	at org.apache.log4j.JBossAppenderHandler.doPublish(JBossAppenderHandler.java:41)
	at org.jboss.logmanager.ExtHandler.publish(ExtHandler.java:79) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:296) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.Logger.logRaw(Logger.java:721) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.Logger.logRaw(Logger.java:731) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.Logger.log(Logger.java:367) [:1.5.2.Final-redhat-1]
	at org.jboss.modules.log.JDKModuleLogger.doLog(JDKModuleLogger.java:109) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.log.JDKModuleLogger.doLog(JDKModuleLogger.java:101) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.log.JDKModuleLogger.trace(JDKModuleLogger.java:129) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:203) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:459) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:408) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:389) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:134) [:1.3.3.Final-redhat-1]
	at org.apache.log4j.JBossAppenderHandler.doPublish(JBossAppenderHandler.java:41)
	at org.jboss.logmanager.ExtHandler.publish(ExtHandler.java:79) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:296) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.Logger.logRaw(Logger.java:721) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.Logger.logRaw(Logger.java:731) [:1.5.2.Final-redhat-1]
	at org.jboss.logmanager.Logger.log(Logger.java:367) [:1.5.2.Final-redhat-1]
	at org.jboss.modules.log.JDKModuleLogger.doLog(JDKModuleLogger.java:109) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.log.JDKModuleLogger.doLog(JDKModuleLogger.java:101) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.log.JDKModuleLogger.trace(JDKModuleLogger.java:129) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:203) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:459) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:408) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:389) [:1.3.3.Final-redhat-1]
	at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:134) [:1.3.3.Final-redhat-1]

Resolution

  • Disable global TRACE logging or TRACE logging for org.jboss.modules

Root Cause

  • This recursion showed that the following happened:
    1. A class is being loaded.
    2. The class loading generates a trace message.
    3. While logging that trace classloader message, the logger needs to load a class.
    4. That next class loading task generates another trace message.
      And then steps 3 and 4 repeat until the thread stack is exhausted
Components
Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.