JBoss - Java heap retention in org.apache.catalina.session.StandardSessions or ClusteredSessions

Environment

• JBoss Enterprise Application Platform
  • 5.x
  • 6.x

Issue

• Our JBoss instance is having OOMEs and/or GC problems and the heap dump shows StandardSessions or ClusteredSessions consuming the majority of the heap

Resolution

• Increase heap sizes to meet demands of session counts and session sizes
• Decrease session timeouts so they are freed from memory more aggressively. Ensure nothing is setting a negative timeout/maxInactiveInterval and persisting sessions in memory indefinitely.
  • What is the session timeout hierarchy in JBoss EAP?
  • If timeouts are set but not apparently honored, see JBoss EAP HTTP Sessions are not expiring or timing out like I would expect
• Call session.invalidate() when appropriate to free a finished session more quickly
• Set a max limit on the number of sessions allowed at once as described in How to configure the maximum number of active sessions allowed in JBoss/Tomcat? to limit the session heap usage
• Cut back on what is added to session attributes to avoid overly large session sizes.
  • If AjaxStateHolders are the largest session attributes, see How to reduce the size of AjaxStateHolder object in the session in JBoss EAP 5?
• Avoid some periodic ping or health-check to the server that would periodically and continously create an empty and unused session. It is best to have the ping request some static content. If the ping requests a JSP page, ensure the session requirement for the health check is disabled as described in How to disable http session creation in specific JSP.

Root Cause

• Large session counts and/or sizes
  • Some client is frequently sending requests without providing back the session generated by JBoss, resulting in an unexpected amount of empty session generation (indicative of some external periodic probe)
  • A health check request is frequently made to a JSP, and JSPs require sessions by default per Why is JBoss creating a new session with every JSP request?
• Overly long session timeouts or infinite timeouts letting inactive sessions persist and build up in memory for too long
• Threading issue is delaying the ContainerBackgroundProcessor so that sessions are not cleared out of memory as expected per timeout settings.
• Large AjaxStateHolders stored in session attributes

Diagnostic Steps

• Troubleshoot with a heap dump as described in Java application "java.lang.OutOfMemoryError: Java heap space" to identify the source of retention.

• OQL to pull out session data:

    SELECT * FROM org.apache.catalina.session.StandardSession
  

• OQL to check all applied session timeouts:

    SELECT s.maxInactiveInterval FROM org.apache.catalina.session.StandardSession s 
  

• OQL to check the most recent access time:

    SELECT s.thisAccessedTime FROM org.apache.catalina.session.StandardSession s 
  

  A value of 0 indicates no reuse after creation. Many such sessions are a likely indication of something continuously hitting the server and creating a new session each time (such as an external ping or health check).

• If a clustered application, look for org.jboss.web.tomcat.service.session.ClusteredSession, FieldBasedClusteredSession, or SessionBasedClusteredSession objects instead.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.