Leak in io.undertow.server.DefaultByteBufferPool on EAP 7.4.16 and 7.4.17

Environment

• JBoss Enterprise Application Platform (EAP)
  • 7.4.16
  • 7.4.17

Issue

• After updating to EAP 7.4.16, we have seen leaks until eventual OOMEs and heap dump show buffers accumulating in io.undertow.server.DefaultByteBufferPool$ThreadLocalCache's localsByThread map:

Class Name                                                                                   | Ref. Objects | Shallow Heap | Ref. Shallow Heap | Retained Heap
---------------------------------------------------------------------------------------------------------------------------------------------------------------
[56] class io.undertow.protocols.ssl.UndertowXnioSsl @ 0xaf1b2a10                            |       71,676 |            8 |         1,720,224 |            32
'- DEFAULT_BUFFER_POOL io.undertow.server.DefaultByteBufferPool @ 0xaf1b2a80                 |       71,676 |           64 |         1,720,224 |    20,792,448
   '- threadLocalCache io.undertow.server.DefaultByteBufferPool$ThreadLocalCache @ 0xaf1b2ac0|       71,676 |           16 |         1,720,224 |    18,071,056
      '- localsByThread java.util.HashMap @ 0xaf1b2ad0                                       |       71,676 |           48 |         1,720,224 |    18,071,040
         '- table java.util.HashMap$Node[131072] @ 0xb3c67d20                                |       71,676 |      524,304 |         1,720,224 |    18,070,992
            |- [108494] java.util.HashMap$Node @ 0xb1610fb0                                  |            6 |           32 |               144 |         1,488
            |  |- next java.util.HashMap$Node @ 0xb1def998                                   |            5 |           32 |               120 |         1,240
            |  |- value io.undertow.server.DefaultByteBufferPool$ThreadLocalData @ 0xb15d5e98|            1 |           24 |                24 |           128
---------------------------------------------------------------------------------------------------------------------------------------------------------------

Resolution

• Update to EAP 7.4.18+
• As a workaround, size task-core-threads equal to task-max-threads to prevent idle thread reclamation that results in this leak. task-max-threads would default to 16*#ofCPUcores if you have not set it so you may set both to such a value to continue with the same thread pool max. For instance, this workaround configured to match the default of a 12 CPU core system:

    <subsystem xmlns="urn:jboss:domain:io:3.0">
        <worker name="default" task-core-threads="192" task-max-threads="192"/>
        <buffer-pool name="default"/>
    </subsystem>

Root Cause

• This content is not included.UNDERTOW-2389
• This content is not included.JBEAP-27062
• This content is not included.JBEAP-27063
• The new ThreadLocalCache implementation in DefaultByteBufferPool does not remove entries upon a thread's release

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.