Spring proxy cache causes PermGen OOME across redeployments

Solution Unverified - Updated

Environment

  • JBoss Enterprise Application Platform (EAP) 6.x
  • Spring

Issue

  • JBoss hits a PermGen OOME after redeploying our spring app many times. It looks like the app classloaders are leaked on redeploy and persisted by a WeakHashMap in org.springframework.cglib.core.AbstractClassGenerator$Source:
org.jboss.modules.ModuleClassLoader @ 0xf3ac5348                                                                      |
'- <classloader> class com.my.custom.Class @ 0xf3ac3808                  |
   '- targetClass org.springframework.aop.target.EmptyTargetSource @ 0xf3db3388                                       |
      '- targetSource org.springframework.aop.framework.AdvisedSupport @ 0xf3db3358                                   |
         '- advised org.springframework.aop.framework.CglibAopProxy$ProxyCallbackFilter @ 0xf3db3340                  |
            '- FIELD_2 org.springframework.cglib.proxy.Enhancer$EnhancerKey$$KeyFactoryByCGLIB$$4ce19e8f @ 0xf3db32f0 |
               '- key java.util.HashMap$Entry @ 0xf3db32d0                                                            |
                  '- [12] java.util.HashMap$Entry[16] @ 0xf3db3280                                                    |
                     '- table java.util.HashMap @ 0xf3db3250                                                          |
                        '- value java.util.WeakHashMap$Entry @ 0xf3db3228                                             |
                           '- next java.util.WeakHashMap$Entry @ 0xf8969c48                                           |
                              '- [54] java.util.WeakHashMap$Entry[64] @ 0xf79780f0                                    |
                                 '- table java.util.WeakHashMap @ 0xf15ae600                                          |
                                    '- cache org.springframework.cglib.core.AbstractClassGenerator$Source @ 0xf15ae570|
                                       '- SOURCE class org.springframework.cglib.proxy.Enhancer @ 0xf15ad538

Resolution

  • Upgrade to Spring 3.2.8
  • A workaround may be to package classes found in the spring proxy cache in a global server module instead of in the application itself. That should prevent spring from having to recache for each redepeployment.

Root Cause

  • The cache is using a WeakHashMap, but the key used is the ModuleClassLoaders. If the ModuleClassLoader is dereferenced, then objects in this WeakHashMap can be freed. The WeakHashMap entries value though ends up chaining a strong reference to the ModuleClassLoader that's used for its key. So this spring caching is improperly using WeakHashMap as a cache since its values reference its keys and prevent any collection from this cache. Thus classloaders are persisted after undeployments by this cache until reaching a PermGen OOME.
  • Content from jira.spring.io is not included.SPR-11276

Diagnostic Steps

  • Capture a heap dump and trace persisted application classloaders to their roots
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.