Distributed transaction fails with "ORA-01013: user requested cancel of current operation" in EAP 6

Environment

• Red Hat JBoss Enterprise Application Platform (EAP)
  • 6.2.4
  • 6.3.x
  • 6.4.x through 6.4.4

Issue

• Two servers are used in a distributed transaction
• The transaction coordinator timeout on both servers is set to a large value (e.g. 300 seconds / 5 minutes)
• An Oracle datasource pool is configured with set-tx-query-timeout set to true
• A transaction begun on server 1 propagates to server 2 where a database operation fails immediately with ORA-01013

java.sql.SQLException: ORA-01013: user requested cancel of current operation
        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)
        at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)
        at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
        at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
        at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
        at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:194)
        at oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:853)
        at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1145)
        at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1259)
        at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1882)
        at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1847)
        at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:301)
        at org.jboss.jca.adapters.jdbc.WrappedStatement.execute(WrappedStatement.java:157)

Resolution

Prior to EAP 6.4.5

• Set set-tx-query-timeout for the datasource pool to false

In EAP 6.4.5 and later as part of This content is not included.BZ-1265300

• The new default (instead of Integer.MAX_VALUE) is 31536000 (one year in seconds).
• The implicit default may be overridden explicitly (on a per-jvm basis) by setting the system property org.jboss.as.ejb3.remote-tx-timeout.

Root Cause

• The timeout period for the propagated transaction branch on server 2 is set to Content from docs.oracle.com is not included.Integer.MAX_VALUE as noted in Parameterizable EJB transaction timeout being overridden ....
  • This is a Content from issues.jboss.org is not included.known issue in EAP 6.4 patch levels earlier than cumulative patch (CP) 5.
• The "inner timeout" value is used to configure the Content from docs.oracle.com is not included.Statement Query Timeout
• The driver or database seems to encounter a "rollover" issue and interprets the large value as a small value

References

• ORA-01013

Diagnostic Steps

Use the Byteman rule below to observe the configured statement query timeout:

RULE WrappedStatementSetQueryTimeout
CLASS org.jboss.jca.adapters.jdbc.WrappedStatement
METHOD setQueryTimeout
IF true
DO traceStack("BYTEMAN - WrappedStatement.setQueryTimeout(" + $1 + ")\n");
ENDRULE

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.