SSL handshakes randomly fail on Java 7

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux (RHEL) 6
    • OpenJDK 7
    • Oracle JRE 1.7
  • JBoss Enterprise Application Platform (EAP)

Issue

  • SSL/TLS handshakes with Diffie-Hellman cipher suites randomly fail from invalid padding exceptions.
  • IOException in getSession(): javax.net.ssl.SSLHandshakeException: Invalid padding
  • javax.net.ssl.SSLHandshakeException: Invalid Padding length: 79 (number may vary)

Resolution

  • As a workaround, use other cipher suites on java 7 (such as RSA). If using JBoss, that can be achieved like so:

      <connector name="https" protocol="HTTP/1.1" scheme="https" socket-binding="https" secure="true">
      <ssl ... cipher-suite="SSL_RSA_WITH_3DES_EDE_CBC_SHA"/>
  </connector>

  • Upgrade to java-1.7.0-openjdk-1.7.0.25-2.3.10.4.el6_4 or Oracle JDK 1.7u51

  • Note that BouncyCastle had a similar error as described in SSL handshakes randomly fail when using BouncyCastle

Root Cause

A defect was introduced in OpenJDK 1.7u6 and Oracle JDK 1.7u17. See:

It has been fixed in OpenJDK u25 and Oracle JDK 1.7u51.

Diagnostic Steps

  • Enable SSL handshake debug logging with the system property below, reproduce, and check for Invalid padding exceptions in DH-cipher handshakes as seen above:

      -Djavax.net.debug=ssl,handshake
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.