JBoss - Java heap retention in session cache DefaultDataContainer after session listener exceptions

Solution Unverified - Updated

Environment

  • JBoss Enterprise Application Platform (EAP) 7.x

Issue

  • We see exceptions during session destruction from a session listener and after enough such exceptions an OutOfMemoryError is eventually reached with heap consumed by many org.infinispan.container.entries.ImmortalCacheEntrys in a org.infinispan.container.impl.DefaultDataContainer session cache
WARN  [org.wildfly.clustering.web.infinispan] (SessionExpirationScheduler - 1) WFLYCLWEBINF0004: Failed to expire session ypieHzHp1gCnVEibsm_ZiYShDaLggbRmYBpa65re: java.lang.RuntimeException: java.lang.NullPointerException
	at io.undertow.servlet//io.undertow.servlet.core.SessionListenerBridge.sessionDestroyed(SessionListenerBridge.java:75)
	at io.undertow.core@2.0.30.SP2-redhat-00001//io.undertow.server.session.SessionListeners.sessionDestroyed(SessionListeners.java:61)
	at org.wildfly.clustering.web.undertow@7.2.8.GA-redhat-00002//org.wildfly.clustering.web.undertow.session.UndertowSessionExpirationListener.sessionExpired(UndertowSessionExpirationListener.java:56)
	at org.wildfly.clustering.web.infinispan@7.2.8.GA-redhat-00002//org.wildfly.clustering.web.infinispan.session.ExpiredSessionRemover.remove(ExpiredSessionRemover.java:61)
	at org.wildfly.clustering.web.infinispan@7.2.8.GA-redhat-00002//org.wildfly.clustering.web.infinispan.session.ExpiredSessionRemover.remove(ExpiredSessionRemover.java:40)
	at org.wildfly.clustering.web.infinispan@7.2.8.GA-redhat-00002//org.wildfly.clustering.web.infinispan.session.SessionExpirationScheduler$ExpirationTask.run(SessionExpirationScheduler.java:149)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
	at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: java.lang.NullPointerException
	at deployment.02721921.war//com.listeners.CustomHttpSessionListener.sessionDestroyed(CustomHttpSessionListener.java:28)
	at io.undertow.servlet//io.undertow.servlet.core.ApplicationListeners.sessionDestroyed(ApplicationListeners.java:328)
	at io.undertow.servlet//io.undertow.servlet.core.SessionListenerBridge.doDestroy(SessionListenerBridge.java:98)
	at io.undertow.servlet//io.undertow.servlet.core.SessionListenerBridge.access$000(SessionListenerBridge.java:41)
	at io.undertow.servlet//io.undertow.servlet.core.SessionListenerBridge$1.call(SessionListenerBridge.java:54)
	at io.undertow.servlet//io.undertow.servlet.core.SessionListenerBridge$1.call(SessionListenerBridge.java:51)
	at io.undertow.servlet//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
	at io.undertow.servlet//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
	at org.wildfly.extension.undertow@7.2.8.GA-redhat-00002//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
	at org.wildfly.extension.undertow@7.2.8.GA-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
	at org.wildfly.extension.undertow@7.2.8.GA-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
	at org.wildfly.extension.undertow@7.2.8.GA-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
	at org.wildfly.extension.undertow@7.2.8.GA-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
	at io.undertow.servlet//io.undertow.servlet.core.SessionListenerBridge.sessionDestroyed(SessionListenerBridge.java:73)
	... 12 more

Resolution

  • Fix any exceptions raised from session listeners
  • Remove the <distributable/> flag from your application WEB-INF/web.xml to disable the distributable cache as a workaround
  • Update to EAP 7.3.4+

Root Cause

Components
Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.