High CPU in XNIO code after updating to EAP 6.4 CP17/CP18

Environment

• Red Hat JBoss Enterprise Application Platform (EAP)
  • 6.4 CP17, CP18

Issue

• After upgrading from EAP 6.4.15.GA to EAP 6.4.17.GA, all our PROD Jboss servers CPU usage went up to 100%.
• We see high CPU in XNIO code like below while connected via SSL enabled management interface:

"Remoting "localhost:MANAGEMENT" read-1" #48 prio=5 os_prio=0 tid=0x00007fb85c0be000 nid=0x1839 runnable [0x00007fb85514b000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
	- locked <0x00000006cf94eb28> (a java.lang.Object)
	at org.xnio.nio.AbstractNioStreamChannel.read(AbstractNioStreamChannel.java:249)
	at org.xnio.ssl.JsseConnectedSslStreamChannel.read(JsseConnectedSslStreamChannel.java:454)
	at org.jboss.remoting3.remote.RemotingMessageChannel.receive(RemotingMessageChannel.java:126)
	- locked <0x00000006cf8cd630> (a java.lang.Object)
	at org.jboss.remoting3.remote.ServerConnectionOpenListener$Initial.handleEvent(ServerConnectionOpenListener.java:243)
	- locked <0x00000006cf8cd600> (a java.util.ArrayDeque)
	at org.jboss.remoting3.remote.ServerConnectionOpenListener$Initial.handleEvent(ServerConnectionOpenListener.java:140)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.channels.TranslatingSuspendableChannel.handleReadable(TranslatingSuspendableChannel.java:189)
	at org.xnio.channels.TranslatingSuspendableChannel$1.handleEvent(TranslatingSuspendableChannel.java:103)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.channels.TranslatingSuspendableChannel.handleReadable(TranslatingSuspendableChannel.java:189)
	at org.xnio.ssl.JsseConnectedSslStreamChannel.handleReadable(JsseConnectedSslStreamChannel.java:183)
	at org.xnio.channels.TranslatingSuspendableChannel$1.handleEvent(TranslatingSuspendableChannel.java:103)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.nio.NioHandle.run(NioHandle.java:90)
	at org.xnio.nio.WorkerThread.run(WorkerThread.java:198)

"Remoting "localhost" read-1" #76 prio=5 os_prio=0 tid=0x00007fb85c137000 nid=0x1855 runnable [0x00007fb85332d000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
	- locked <0x00000006cf951548> (a java.lang.Object)
	at org.xnio.nio.AbstractNioStreamChannel.read(AbstractNioStreamChannel.java:249)
	at org.jboss.remoting3.remote.RemotingMessageChannel.readMessageLength(RemotingMessageChannel.java:78)
	at org.jboss.remoting3.remote.ServerConnectionOpenListener$Initial.handleEvent(ServerConnectionOpenListener.java:228)
	at org.jboss.remoting3.remote.ServerConnectionOpenListener$Initial.handleEvent(ServerConnectionOpenListener.java:140)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.channels.TranslatingSuspendableChannel.handleReadable(TranslatingSuspendableChannel.java:189)
	at org.xnio.channels.TranslatingSuspendableChannel$1.handleEvent(TranslatingSuspendableChannel.java:103)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.nio.NioHandle.run(NioHandle.java:90)
	at org.xnio.nio.WorkerThread.run(WorkerThread.java:198)

Resolution

• Update to EAP 6.4 CP19 or later

Root Cause

• Remoting was not guarding against zero length messages, which results in repeated attempts to read into a zero length buffer.
• CVE-2018-1041

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.