"The connection is closed" / "Closed Connection" exception raised by JDBC driver deployed in JBoss EAP

Environment

• Red Hat JBoss Enterprise Application Platform
  • 8
  • 7
  • 6
  • 5
• All databases

Issue

• JDBC driver raises an exception like the below indicating connection closure:

  ####Microsoft SQL Server

  ... com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
      ...
      at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:...)
      at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:...)
      at com.microsoft.sqlserver.jdbc.SQLServerStatement.checkClosed(SQLServerStatement.java:...)
  

  ... com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
      at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:...)
      at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:...)
      at com.microsoft.sqlserver.jdbc.SQLServerConnection.prepareStatement(SQLServerConnection.java:...)
  

  ####Oracle

  ... java.sql.SQLRecoverableException: Closed Connection
      at oracle.jdbc.driver.PhysicalConnection.createStatement(PhysicalConnection.java:3423)
      ...
  

  ... java.sql.SQLRecoverableException: Closed Connection
      at oracle.jdbc.driver.PhysicalConnection.setAutoCommit(PhysicalConnection.java:2232)
      ...
  

  ...  java.sql.SQLException: Closed Connection
      ...
      at oracle.jdbc.driver.PhysicalConnection.prepareStatement(PhysicalConnection.java:868)
  

  ####PostgreSQL

  ...  org.postgresql.util.PSQLException: Connection has been closed.
       at org.postgresql@xx.xx.xx//org.postgresql.ds.PGPooledConnection$ConnectionHandler.invoke(PGPooledConnection.java:326)
       ...
  

Resolution

One or more of the following may be relevant depending on the scenario in which the connection has been externally terminated.

• Configure the JBoss EAP idle-timeout-minutes datasource parameter to a time period smaller than the timeout configured within the database server. Note that this timeout configuration applies only to connections which are not owned by application components.
• Ensure that fault tolerance/validation¹ is configured. Typically, validate-on-match is recommended which would ensure that only valid connections are returned in response to getConnection() calls. Again, note that this configuration is relevant only to connections which are not owned by application components.
• Review the performance of the system to ensure that application threads are not running for longer periods than expected and that no unexpected blocking occurs. Either of these scenarios could result in connections owned by application components sitting unused for long periods. Note that connections taken from the pool by application components are considered by JBoss EAP to be InUse (owned by the application component which took them from the pool) and are not subject to EAP-initiated timeout (they may, of course, be externally timed out/terminated by the network or database server) nor are they subject to JBoss fault tolerance/validation facilities.
• If the termination occurs while the connection is held by an application component, this may indicate that the application needs to be refactored to utilize connections more efficiently (i.e. connections are not meant to be held for extended periods - they should be taken from the pool, used and returned quickly as long running transactions are an anti-pattern). If the application code cannot be refactored for efficient use of connections, it may be necessary to increase the timeout periods configured in the database server. Consult your DBA and/or the JDBC driver vendor's support for further information/recommendations.

NOTE When configuring pool flush strategies care has to be take as to which strategy should be used. Strategies like EntirePool or AllConnections could result in active connections being terminated. For that reason they are not suitable for production systems.

Root Cause

• The exception is raised by the JDBC driver to indicate that something outside of JBoss terminated the connection to the database server.
  • This termination originated externally (e.g. due to network or database server initiated activity), not from within JBoss.
• The error might be likely to occur in the following cases
  • A connection sits unused in the EAP (idle) pool for a period of time and fault tolerance/validation is not enabled.
  • A connection is taken from the pool by an application component and left idle for a period of time due to the way in which the application logic is factored.
  • A performance problem or blocking slows the work of a thread holding a connection and the connection remains idle for a period of time.
  • A long running SQL command fails to receive a response from the database server for an extended period of time.

Diagnostic Steps

• Verify that the correct JDBC driver/version is used
• Verify that the database server is online, accepting traffic and not under high load
  • Examine database logs for indications of errors
• Verify that the database server timeout period is configured appropriately (e.g. is not set to time out connections too quickly)
• Verify network connectivity/status (e.g. check network connection timeouts, firewall configuration, etc.)
  • Examine network logs for indications of connectivity failures
• Determine if there is a pattern to the incidents
  • Errors after an idle period on the server may suggest an external timeout
  • Errors at specific times of day over the course of a few days may indicate some load issue or periodic administrative action

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.