Threads Blocked in org.jboss.web.tomcat.service.WebCtxLoader$ENCLoader

Environment

• Red Hat JBoss Enterprise Application Platform (EAP)
  • 5.0.x

Issue

• Thread dumps on the application showed that most of the application threads used for the business functionality were blocked by a single lock: BLOCKED on org.jboss.web.tomcat.service.WebCtxLoader$ENCLoader. For example:

"http-localhost-8080-13" - Thread t@280
   java.lang.Thread.State: BLOCKED on org.jboss.web.tomcat.service.WebCtxLoader$ENCLoader@5050c52b owned by: http-host1.example.com%2F199.81.100.127-7583-16
    at java.lang.ClassLoader.loadClass(ClassLoader.java:292)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    at org.apache.xerces.parsers.ObjectFactory.findProviderClass(Unknown Source)
    at org.apache.xerces.parsers.ObjectFactory.newInstance(Unknown Source)
    at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
    ...

Resolution

Update to JBoss EAP 5.1 or greater.

Or install the appropriate patches:

• EAP 5.0.1: <This content is not included.https://access.redhat.com/jbossnetwork/restricted/softwareDetail.html?softwareId=3273>
• EAP 5.0.0: <This content is not included.https://access.redhat.com/jbossnetwork/restricted/softwareDetail.html?softwareId=2143>

Root Cause

• Threads are blocked waiting to access the synchronized method java.lang.ClassLoader.loadClass(String name, boolean resolve).
• <Content from issues.jboss.org is not included.https://issues.jboss.org/browse/JBPAPP-4246>
• <Content from issues.jboss.org is not included.https://issues.jboss.org/browse/JBPAPP-4045>. This may only be a contributing factor, not a direct cause; however, the stack trace associate with this issue has been observed related to this issue.

Diagnostic Steps

• Review classloading configuration.
• Ask for boot.log to see if there are any environment variables being set to override the default parser (-D ...TransformationFactory).
• Get a listing of their deployments (e.g. tree app.war).
• Check if blocking is not because of CPU starvation.
• Could be due to huge hike in number of request (for example, load testing).

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.