JVM Crashes with EXCEPTION_STACK_OVERFLOW using OpenJDK on Windows

Environment

• Windows
• OpenJDK
  • 8
  • 11

Issue

• OpenJDK 11 Red Hat build - Fatal error has been detected by the Java Runtime Environment: EXCEPTION_STACK_OVERFLOW

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_STACK_OVERFLOW (0xc00000fd) at pc=0x000007fef5f571b8, pid=2480, tid=4472
#
# JRE version: OpenJDK Runtime Environment (11.0.4+11) (build 11.0.4-redhat+11-LTS)
# Java VM: OpenJDK 64-Bit Server VM (11.0.4-redhat+11-LTS, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C  [nio.dll+0x71b8]
...
Stack: [0x0000000045220000,0x0000000045320000],  sp=0x000000004522cc78,  free space=51k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [nio.dll+0x71b8]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 12597  sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(JI[I[I[IJ)I java.base@11.0.4-redhat (0 bytes) @ 0x0000000011078652 [0x00000000110785c0+0x0000000000000092]
J 12799 c1 sun.nio.ch.WindowsSelectorImpl$SubSelector.poll()I java.base@11.0.4-redhat (47 bytes) @ 0x0000000009f1f294 [0x0000000009f1f160+0x0000000000000134]
J 9991 c1 sun.nio.ch.WindowsSelectorImpl.doSelect(Ljava/util/function/Consumer;J)I java.base@11.0.4-redhat (151 bytes) @ 0x000000000aeec13c [0x000000000aeebd40+0x00000000000003fc]
J 9990 c1 sun.nio.ch.SelectorImpl.lockAndDoSelect(Ljava/util/function/Consumer;J)I java.base@11.0.4-redhat (87 bytes) @ 0x000000000aee7374 [0x000000000aee6f40+0x0000000000000434]
J 16573 c1 sun.nio.ch.SelectorImpl.select()I java.base@11.0.4-redhat (9 bytes) @ 0x000000000b8fd7ac [0x000000000b8fd740+0x000000000000006c]
j  org.xnio.nio.SelectorUtils.await(Lorg/xnio/nio/NioXnio;Ljava/nio/channels/SelectableChannel;I)V+40
j  org.xnio.nio.NioSocketConduit.awaitWritable()V+26
j  org.xnio.conduits.AbstractSinkConduit.awaitWritable()V+7
j  io.undertow.conduits.ChunkedStreamSinkConduit.awaitWritable()V+7
j  org.xnio.conduits.ConduitStreamSinkChannel.awaitWritable()V+4
j  io.undertow.channels.DetachableStreamSinkChannel.awaitWritable()V+20
j  io.undertow.server.HttpServerExchange$WriteDispatchChannel.awaitWritable()V+20

Resolution

• Increase -XX:StackShadowPagesflag value from 7 (default) to 12 as a workaround.
• Upgrade to 11.0.5+ or 1.8.0.232+

Root Cause

This occurs because the JVM has a dedicated space for native calls. Stacks from native calls are stored in ShadowPages, if the pages aren't enough, the error occurs and on Windows the JVM crashes.

Check more information about it on Content from bugs.openjdk.java.net is not included.JDK-8216472.

Diagnostic Steps

Check this solution.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.