JBoss unresponsive due to http connector threads blocked in org.apache.naming.resources.ProxyDirContext.cacheLoad()

Solution Verified - Updated

Environment

  • JBoss AS 4.0.x

Issue

  • When performance deterioriates, a jnlp file (that launches a Swing client on a client machine) cannot be downloaded.
  • Many http connector threads blocked with the following stack trace:
    Thread: http-0.0.0.0-8881-35 : priority:5, demon:true, threadId:164, threadState:BLOCKED, threadLockName:org.apache.nam
org.apache.naming.resources.ProxyDirContext.cacheLoad(ProxyDirContext.java:1597)
org.apache.naming.resources.ProxyDirContext.cacheLookup(ProxyDirContext.java:1448)
org.apache.naming.resources.ProxyDirContext.lookup(ProxyDirContext.java:282)
org.apache.catalina.core.ApplicationContext.getResource(ApplicationContext.java:498)
org.apache.catalina.core.ApplicationContextFacade.getResource(ApplicationContextFacade.java:178)
jnlp.sample.servlet.JnlpResource.(JnlpResource.java:135)
jnlp.sample.servlet.JnlpResource.(JnlpResource.java:108)
jnlp.sample.servlet.JnlpResource.(JnlpResource.java:97)
jnlp.sample.servlet.JnlpDownloadServlet.handleBasicDownload(JnlpDownloadServlet.java:224)
jnlp.sample.servlet.JnlpDownloadServlet.locateResource(JnlpDownloadServlet.java:211)
jnlp.sample.servlet.JnlpDownloadServlet.handleRequest(JnlpDownloadServlet.java:139)
jnlp.sample.servlet.JnlpDownloadServlet.doGet(JnlpDownloadServlet.java:113)
javax.servlet.http.HttpServlet.service(HttpServlet.java:697)
javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
  • Thread dumps show the number of http connector threads is at the maximum value, the Connector maxThreads in server.xml.

Resolution

Set Context cachingAllowed="false". For example:

<Context cookies="true" crossContext="true" cachingAllowed="false">
</Context>

Root Cause

ProxyDirContext.cacheLoad() is synchronized:

// Add new entry to cache
synchronized (cache) {
    // Check cache size, and remove elements if too big
    if (cache.allocate(entry.size)) {
        cache.load(entry);
    }
...

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.