Java/JBoss hangs creating and authenticating connections to the database
Environment
- JBoss Enterprise Application Platform (EAP)
- SunJDK 1.6.0_29
- OpenJDK
- 1.6.0.0-1.23.1.9.10.el5_7
- 1.6.0.0-1.40.1.9.10.el6_1
- OpenJDK versions reported as "1.6.0_22,Sun Microsystems Inc." in the boot.log
- MS SQL 2008
Issue
- Can not Connect to database (DB) after patching RHEL via yum. The JBoss server is unable connected to our Microsoft SQL 2008 R2 database server. Our unpatched servers do not have this issue.
- Our JBoss server/Java application hangs. Looks like this started after a JDK upgrade/yum update. Thread dumps show threads hanging with stack traces like the following:
"main" prio=10 tid=0x00002aaab00ad000 nid=0x5d40 runnable [0x000000004045a000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:146) at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1647) at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:3694) - locked <0x00000000ef9f7d78> (a com.microsoft.sqlserver.jdbc.TDSReader) at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:5026) at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:4978) at com.microsoft.sqlserver.jdbc.SQLServerConnection$1ConnectionCommand.doExecute(SQLServerConnection.java:1441) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1400) - locked <0x00000000ef7bfae0> (a java.lang.Object) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(SQLServerConnection.java:1446) at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:1907) at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:41) at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:1893) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1400) - locked <0x00000000ef7bfae0> (a java.lang.Object) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1045) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:817) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:700) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:842) at org.jboss.resource.adapter.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:207) at org.jboss.resource.adapter.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:195) at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.createConnectionEventListener(InternalManagedConnectionPool.java:639) at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.java:273) at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool.java:690) at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:404) at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:424) at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:496) at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:941) at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:89)
Resolution
-
Upgrade to JDK 1.6 update 30
-
Revert to a JDK 1.6 update prior to update 29
-
Try the workaround referenced in java-1.6.0-openjdk and java-1.6.0-sun security updates introduce a regression bug affecting JDBC connections to Microsoft SQL Server
-
Engage the database vendor for any specific solutions they may have to offer
Root Cause
- Threads are hanging while creating and authenticating connections to the database
- Others have reported such behavior and tracked it down to a JDK bug introduced in JDK 1.6.0_29
- JVM bug introduced when it was updated on the system.
Diagnostic Steps
- Troubleshoot using thread dumps and additional steps mentioned in Java application unresponsive
- Was the JDK updated recently?
- This should be reproducible outside of JBoss and more complicated Java applications. In a simple test scenario with a class that solely opens a connection to the database, does it come to hang in a similar manner?
- Check for any known issues specific to the database and driver version in use.
Product(s)
Category
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.