Shutdown or undeployment hangs in DistributableSessionManager.stop until HTTP requests finish processing

Environment

• JBoss Enterprise Application Platform (EAP) 6.x

Issue

• In JBoss EAP 6, If application is in use(web connector threads are in process, http sessions active), we can not redeploy application. Thread dumps show it stalled in DistributableSessionManager.stop:

"ServerService Thread Pool -- 135" prio=6 tid=0x0000000045bfb000 nid=0x22ac waiting on condition [0x000000005e88e000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000404554f28> (a java.util.concurrent.Semaphore$FairSync)
	at java.util.concurrent.locks.LockSupport.park(Unknown Source)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(Unknown Source)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(Unknown Source)
	at java.util.concurrent.Semaphore.acquire(Unknown Source)
	at org.jboss.as.web.session.DistributableSessionManager.stop(DistributableSessionManager.java:419)
	at org.apache.catalina.core.StandardContext.stop(StandardContext.java:3916)

Resolution

• With EAP 6.3.0+, set the jboss.web.session.manager.stop.timeout system property to a desired value to limit this wait (in milliseconds)
• Upgrade to latest EAP 7.4. This Content from github.com is not included.pulls the distributable session manager stop timeout from the cache stop timeout. This defaults to 10000 ms but can be adjusted on the web cache transaction stop-timeout:

            <cache-container name="web" default-cache="dist" modules="org.wildfly.clustering.web.infinispan">
                <distributed-cache name="dist">
                    <locking isolation="REPEATABLE_READ"/>
                    <transaction mode="BATCH" stop-timeout="#ofMS"/>
                    <file-store/>
                </distributed-cache>
            </cache-container>

Root Cause

• This content is not included.BZ-998779
• When JBoss undeploys an application (which is the first half of a re-deployment), it pauses to wait for all the HTTP requests being processed to finish so that no sessions are in use before stopping the session manager.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.