JVM crash in java.util.zip.Deflater.deflateBytes
Environment
- Linux
- OpenJDK
- 7
- 8
- Tomcat 7 with
org.apache.coyote.http11.Http11Protocol.COMPRESSIONenabled
Issue
The JVM crashes with one of the following in the fatal error log:
# SIGSEGV (0xb) at pc=0x0000003103e07cb4, pid=28368, tid=140632258549504
#
# JRE version: OpenJDK Runtime Environment (7.0_75-b13) (build 1.7.0_75-mockbuild_2015_01_08_20_32-b00)
# Java VM: OpenJDK 64-Bit Server VM (24.75-b04 mixed mode linux-amd64 compressed oops)
# Derivative: IcedTea 2.5.4
# Distribution: Built on Red Hat Enterprise Linux Server release 6.6 (Santiago) (Thu Jan 8 20:32:29 EST 2015)
# Problematic frame:
# C [libz.so.1+0x7cb4] _tr_stored_block+0x144
...
Current thread (0x00007fe748002000): JavaThread "http-bio-127.4.253.1-8080-exec-1" daemon [_thread_in_native, id=32307, stack(0x00007fe77fc89000,0x00007fe77fd0a000)]
...
Stack: [0x00007fe77fc89000,0x00007fe77fd0a000], sp=0x00007fe77fd012b0, free space=480k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libz.so.1+0x7cb4] _tr_stored_block+0x144
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j java.util.zip.Deflater.deflateBytes(J[BIII)I+0
j java.util.zip.Deflater.deflate([BIII)I+84
j java.util.zip.Deflater.deflate([BII)I+5
J 6744 C2 java.util.zip.DeflaterOutputStream.write([BII)V (88 bytes) @ 0x00007fe788c8bf98 [0x00007fe788c8bca0+0x2f8]
j java.util.zip.GZIPOutputStream.write([BII)V+4
j org.apache.coyote.http11.filters.FlushableGZIPOutputStream.flushLastByte()V+23
j org.apache.coyote.http11.filters.FlushableGZIPOutputStream.flush()V+26
j org.apache.coyote.http11.filters.GzipOutputFilter.flush()V+32
j org.apache.coyote.http11.AbstractOutputBuffer.flush()V+92
j org.apache.coyote.http11.InternalOutputBuffer.flush()V+1
j org.apache.coyote.http11.AbstractHttp11Processor.action(Lorg/apache/coyote/ActionCode;Ljava/lang/Object;)V+130
j org.apache.coyote.Response.action(Lorg/apache/coyote/ActionCode;Ljava/lang/Object;)V+31
j org.apache.catalina.connector.OutputBuffer.doFlush(Z)V+97
j org.apache.catalina.connector.OutputBuffer.flush()V+2
j org.apache.catalina.connector.CoyoteWriter.flush()V+12
...
Stack: [0x00007f6400cd2000,0x00007f6400dd3000], sp=0x00007f6400dd0530, free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libzip.so+0x45be] fill_window+0x17e
C [libzip.so+0x492d] deflate_slow+0x19d
C [libzip.so+0x6212] deflate+0x2a2
C [libzip.so+0x3012] Java_java_util_zip_Deflater_deflateBytes+0x212
J 3602 java.util.zip.Deflater.deflateBytes(J[BIII)I (0 bytes) @ 0x00007f641d41accd [0x00007f641d41ac00+0xcd]
J 3706 C2 java.util.zip.Deflater.deflate([BIII)I (140 bytes) @ 0x00007f641d12bfc4 [0x00007f641d12bec0+0x104]
J 3627 C1 com.soluzionasf.onis.util.compress.OnisDeflaterOutputStream.deflate()V (40 bytes) @ 0x00007f641d6c447c [0x00007f641d6c4160+0x31c]
J 3625 C1 java.util.zip.DeflaterOutputStream.write([BII)V (88 bytes) @ 0x00007f641d3f3a04 [0x00007f641d3f3160+0x8a4]
...
Resolution
OpenJDK
A workaround is to disable mmap usage by setting Dsun.zip.disableMemoryMapping=true or simply -Dsun.zip.disableMemoryMapping.
The real fix is to address the application bug or invalid operation that modifies a file while it is in use.
Tomcat 7
A workaround would be to offload the compression to httpd:
<Location /APP_CONTEXT/>
SetOutputFilter DEFLATE
Header append Vary User-Agent env=!dont-vary
</Location>
Root Cause
A zip or jar file that is in use is being overwritten.
It could also be a bug:
This content is not included.This content is not included.https://bugzilla.redhat.com/1375358
Diagnostic Steps
See Java application down due to JVM crash.
Update to the latest zlib and OpenJDK versions.
Test with -Dsun.zip.disableMemoryMapping=true.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.