Deadlock in AbstractFramedChannel in the JBoss EAP 7.1

Environment

Red Hat JBoss Enterprise Application Platform (EAP) 7.1

Issue

• We have problems with JBoss, because on several occasions a week, CPU consumption goes up making Http services unnacessible.

Found one Java-level deadlock:

"default task-28":
  waiting to lock monitor 0x000000000daeca28 (object 0x000000074bea12d0, a io.undertow.protocols.http2.Http2Channel),
  which is held by "default I/O-3"
"default I/O-3":
  waiting to lock monitor 0x000000000b2c50e8 (object 0x000000074bfeeb20, a java.lang.Object),
  which is held by "default task-28"

Java stack information for the threads listed above:

"default task-28":
	at io.undertow.server.protocol.framed.AbstractFramedChannel.notifyClosed(AbstractFramedChannel.java:904)
	- waiting to lock <0x000000074bea12d0> (a io.undertow.protocols.http2.Http2Channel)
	at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel.exitRead(AbstractFramedStreamSourceChannel.java:584)
	- locked <0x000000074bfeeb20> (a java.lang.Object)
	at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel.read(AbstractFramedStreamSourceChannel.java:531)
	at io.undertow.protocols.http2.Http2StreamSourceChannel.read(Http2StreamSourceChannel.java:141)
	at org.xnio.conduits.StreamSourceChannelWrappingConduit.read(StreamSourceChannelWrappingConduit.java:43)
	at org.xnio.conduits.ConduitStreamSourceChannel.read(ConduitStreamSourceChannel.java:127)
	at io.undertow.channels.DetachableStreamSourceChannel.read(DetachableStreamSourceChannel.java:209)
	at io.undertow.server.HttpServerExchange$ReadDispatchChannel.read(HttpServerExchange.java:2301)
	at io.undertow.servlet.spec.ServletInputStreamImpl.readIntoBufferNonBlocking(ServletInputStreamImpl.java:198)
	at io.undertow.servlet.spec.ServletInputStreamImpl.available(ServletInputStreamImpl.java:231)
	at java.io.FilterInputStream.available(FilterInputStream.java:168)
	at java.io.PushbackInputStream.available(PushbackInputStream.java:277)
	at sun.nio.cs.StreamDecoder.inReady(StreamDecoder.java:363)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	- locked <0x000000074c195ba8> (a java.io.InputStreamReader)
        ...

"default I/O-3":
	at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel.markStreamBroken(AbstractFramedStreamSourceChannel.java:657)
	- waiting to lock <0x000000074bfeeb20> (a java.lang.Object)
	at io.undertow.protocols.http2.AbstractHttp2StreamSourceChannel.markStreamBroken(AbstractHttp2StreamSourceChannel.java:68)
	at io.undertow.server.protocol.framed.AbstractFramedChannel.markReadsBroken(AbstractFramedChannel.java:823)
	at io.undertow.server.protocol.framed.AbstractFramedChannel.receive(AbstractFramedChannel.java:474)
	- locked <0x000000074bea12d0> (a io.undertow.protocols.http2.Http2Channel)
        ...

Resolution

Fixed in EAP 7.2 or Apply JBoss EAP 7.1 CP6

Root Cause

Content from issues.jboss.org is not included.JBEAP-15330 - UNDERTOW-1231/UNDERTOW-1179 - Deadlock AbstractFramedChannel when notifyClosed and markStreamBroken are called related to Http2Channel

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.