Deadlock in JBoss' StatefulSessionFilePersistenceManager when passivating stateful session beans

Solution Verified - Updated 7 Aug 2024

Environment

JBoss Enterprise Application Platform (EAP) 5.0.0 to 5.1.2

Issue

Our JBoss instance reported a deadlock in JBoss EJB code, when one thread was accessing a passivated SFSB and another thread was creating a new SFSB instance.

"WorkerThread#1[127.0.0.1:49210]":
        at org.jboss.ejb3.cache.simple.SimpleStatefulCache.get(SimpleStatefulCache.java:472)
        - waiting to lock <0x00000000fdba4c68> (a org.jboss.ejb3.cache.simple.SimpleStatefulCache$CacheMap)
        - locked <0x00000000fdba5cd8> (a org.jboss.ejb3.cache.simple.StatefulSessionFilePersistenceManager)
        at org.jboss.ejb3.cache.simple.SimpleStatefulCache.get(SimpleStatefulCache.java:430)
        at org.jboss.ejb3.stateful.StatefulInstanceInterceptor.invoke(StatefulInstanceInterceptor.java:59)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.aspects.tx.TxPolicy.invokeInOurTx(TxPolicy.java:79)
        at org.jboss.aspects.tx.TxInterceptor$Required.invoke(TxInterceptor.java:190)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:76)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.ejb3.security.RoleBasedAuthorizationInterceptorv2.invoke(RoleBasedAuthorizationInterceptorv2.java:201)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.ejb3.security.Ejb3AuthenticationInterceptorv2.invoke(Ejb3AuthenticationInterceptorv2.java:182)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:41)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.ejb3.BlockContainerShutdownInterceptor.invoke(BlockContainerShutdownInterceptor.java:67)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.ejb3.core.context.CurrentInvocationContextInterceptor.invoke(CurrentInvocationContextInterceptor.java:47)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.aspects.currentinvocation.CurrentInvocationInterceptor.invoke(CurrentInvocationInterceptor.java:67)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.ejb3.interceptor.EJB3TCCLInterceptor.invoke(EJB3TCCLInterceptor.java:86)
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
        at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:569)
        at org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
        at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
        at org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
        at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:967)
        at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:791)
        - locked <0x00000000f1f4dcf0> (a org.jboss.remoting.transport.socket.ServerThread)
        at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:744)
        at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:548)
        at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)
"WorkerThread#0[127.0.0.1:65439]":
        at org.jboss.ejb3.cache.simple.SimpleStatefulCache.passivate(SimpleStatefulCache.java:382)
        - waiting to lock <0x00000000fdba5cd8> (a org.jboss.ejb3.cache.simple.StatefulSessionFilePersistenceManager)
        at org.jboss.ejb3.cache.simple.SimpleStatefulCache$CacheMap.removeEldestEntry(SimpleStatefulCache.java:96)
        - locked <0x00000000f353a9c0> (a org.jboss.ejb3.stateful.StatefulBeanContext)
        at java.util.LinkedHashMap.addEntry(LinkedHashMap.java:410)
        at java.util.HashMap.put(HashMap.java:385)
        at org.jboss.ejb3.cache.simple.SimpleStatefulCache.create(SimpleStatefulCache.java:409)
        - locked <0x00000000fdba4c68> (a org.jboss.ejb3.cache.simple.SimpleStatefulCache$CacheMap)
        at org.jboss.ejb3.stateful.StatefulContainer.createSession(StatefulContainer.java:406)
        at org.jboss.ejb3.session.SessionContainer.createSession(SessionContainer.java:673)
 
       at 
org.jboss.ejb3.proxy.impl.factory.session.stateful.StatefulSessionProxyFactoryBase.getNewSessionId(StatefulSessionProxyFactoryBase.java:276)
 
       at 
org.jboss.ejb3.proxy.impl.factory.session.stateful.StatefulSessionProxyFactoryBase.createProxyBusiness(StatefulSessionProxyFactoryBase.java:140)
        at sun.reflect.GeneratedMethodAccessor318.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:121)
        at org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
        at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:967)
        at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:791)
        - locked <0x00000000f170e008> (a org.jboss.remoting.transport.socket.ServerThread)
        at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:744)
        at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:548)
        at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)

Resolution

Upgrade to EAP 5.1.3 or install the one-off patch for JBPAPP-7523 (currently available for EAP 5.1.1).

Root Cause

The issue JBPAPP-7523 caused a deadlock in the EJB container code if one thread created a new SFSB instance which required passivating an existing one (due to limits) while another thread was activating a passivated SFSB.

Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.