OutOfMemoryError with CXF BusFactory as top suspect

Environment

• JBoss EAP 6.1
• CXF API 2.6.6
• JBOSS WS 4.1.3

Issue

OutOfMemoryError with CXF BusFactory as top suspect

Resolution

This does look like something that can be avoided by the way the clients are handled.

Generally speaking, the user needs to be aware of the Apache CXF Bus implications of creating JAXWS clients in-container. Each time a client is created, a Bus is needed to create the client in. By default, if there's no Bus already associated to the current thread, a new bus will be created and linked to the current thread. This means the user can eventually end up with N busses, where N is the size of thread pool serving the web, ejb3 or whatever requests. This is usually relevant when the clients that are created are based on big wsdl docs with multiple and perhaps deeply nested xsd, because the WSDL4J model for the docs consumes a lot of memory. The wsdl model is not cleared up if the client proxy is garbage collected, it's stored in a cache class in the bus, so again it has the same lifecycle of the bus.

In summary, the user has to take actions to prevent polluting the system with multiple unneeded bus instances; the possible solutions are:

1. Re-use / pool clients

2. Destroy the bus after use or at least cleanup it from the thread association (requires direct usage of CXF API, Bus and BusFactory). This can be done like so:

    try {
      Service service = Service.create(wsdlURL, serviceQName);
      MyEndpoint port = service.getPort(MyEndpoint.class);
      //...
    } finally {
      Bus bus = BusFactory.getThreadDefaultBus(false);
      if (bus != null)
        bus.shutdown(true);
    }
   

   When using this approach, add a jboss-deployment-structure.xml to your WAR's WEB-INF/ or EAR's META-INF/ with the following:

    <jboss-deployment-structure>
      <deployment>
        <dependencies>
          <module name="org.apache.cxf"/>
          <module name="org.apache.cxf.impl"/>
        </dependencies>
      </deployment>
    </jboss-deployment-structure>
   

3. Use a proper Bus Selection Strategy (requires )JBossWS_ API and JBossWS 4.2.2 / JBoss EAP 6.2): it might be a good idea to evaluate using TCCL_BUS bus strategy when creating the clients; that would cause a single bus for each classloader to be created.

Root Cause

Looking at the heap dump analysis, the top object is org.apache.cxf.BusFactory. Almost all of this data is retained in one WeakHashMap member. Looking in the source for Content from anonsvn.jboss.org is not included.BusFactory, this is that member:

    protected static Map<Thread, Bus> threadBusses = new WeakHashMap<Thread, Bus>();

So it's the thread-local buses that is being retained (probably one per thread). In a Content from docs.oracle.com is not included.WeakHashMap, it's the key that has the weak reference, not the value. Since the Thread objects are most likely permanently held in the JBossWeb thread pool, the corresponding Bus values are never removed by the garbage collector.

Diagnostic Steps

A JAX-WS proxy should only create one instance of a Bus and all the corresponding data, but in the suspected behavior it is possible to see a Bus replicated for every Thread.

Under normal circumstances the Bus objects are removed by JBossWS, but for some reason this is not happening in this situation.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.