Heap is fully consumed by BouncyCastle objects

Environment

• Red Hat JBoss Enterprise Application Platform 6
• BouncyCastle 1.47

Issue

• Out of memory errors are seen in JBoss instances.
• Examining the heap dump shows the majority of memory is consumed by BouncyCastle:

Class Name                                        | Objects | Shallow Heap | Retained Heap
-------------------------------------------------------------------------------------------
org.bouncycastle.jce.provider.BouncyCastleProvider|   5,440 |      522,240 | 1,601,020,200
-------------------------------------------------------------------------------------------

Resolution

• This is a known issue in BouncyCastle documented at Content from www.bouncycastle.org is not included.memory leak at deprecated method "PGPEncryptedDataGenerator.addMethod". Ensure you have upgraded bouncy castle to acquire the fix.
• If using a BouncyCastleProvider directly in your application code, create the provider once and cache it for reuse instead of recreating the provider repeatedly.

Root Cause

• There is a memory leak inside of BouncyCastle in PGPEncryptedDataGenerator.addMethod.
• The BouncyCastleProvider is being created repeatedly; recreating a security provider can cause a leak per Heap growth from security providers stored in javax.crypto.SunJCE_b

Diagnostic Steps

• Obtained a heap dump from the time of the issue as described in How do I create a Java heap dump?
• Examined this and discovered a large HashMap of multiple BouncyCastle objects.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.