Application classloaders leaked by custom JCA provider reference in sun.security.jca.ProviderList

Environment

• JBoss Enterprise Application Platform (EAP)

Issue

• We're experiencing a metaspace leak across application redeployments. A heap dump shows classloaders persisted by references from the sun.security.jca.ProviderList to a custom JCA provider in the application:

Class Name                                                                       | Ref. Objects | Shallow Heap | Ref. Shallow Heap | Retained Heap
---------------------------------------------------------------------------------------------------------------------------------------------------
class sun.security.jca.Providers @ 0x663b96b98 System Class                      |            1 |           24 |                88 |           248
'- providerList sun.security.jca.ProviderList @ 0x674178708                      |            1 |           24 |                88 |       121,752
   '- configs sun.security.jca.ProviderConfig[39] @ 0x674178720                  |            1 |          176 |                88 |       121,704
      '- [38] sun.security.jca.ProviderConfig @ 0x6741787d0                      |            1 |           32 |                88 |       121,528
         '- provider my.custom.provider.Class @ 0x6741787f0                   |            1 |           96 |                88 |       121,496
            '- <class> class my.custom.provider.Class @ 0x674225980           |            1 |           40 |                88 |         7,984
               '- <classloader> org.jboss.modules.ModuleClassLoader @ 0x673fe5928|            1 |           88 |                88 |     2,055,248

Resolution

• Instead of providing the JCA provider in the app level like this, you could provide it at the server level per Using a custom JCA provider in Java 11 and JBoss EAP 7.
• Otherwise if you provide it at the app level, you should call java.security.Security.removeProvider upon application undeployment to remove the custom provider and avoid leaks from it.

Root Cause

• Application is providing and adding but not removing a custom JCA provider.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.