How to configure an EAP6 ejb-client to fail fast if the network connection fail

Solution Verified - Updated

Environment

  • Red Hat JBoss Enterprise Application Platform (EAP)
    • 6.x

Issue

  • My ejb client is blocked for a long time if the network connection failed, how can I avoid this?
  • I use a UNIX command to simulate a network failure and my clients stuck, how can I configure a faster failure?
iptables -P INPUT DROP ; iptables -P FORWARD DROP ; iptables -P OUTPUT DROP
  • The ejb client hang for longer and the configuration of org.xnio.Options.READ_TIMEOUT and org.xnio.Options.WRITE_TIMEOUT does not help, which property need to be configured?
  • To test the client failover behaviour a network administrator shuts down the switch port of one machine.
    Afterwards the client tries to call a remote function which fails with an error after waiting for the configured invocation-timeout.
Caused by: javax.ejb.EJBException: java.util.concurrent.TimeoutException: No invocation response received in 2000 milliseconds
	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:236) [jboss-ejb-client-1.0.24.Final-redhat-1.jar:1.0.24.Final-redhat-1]
	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:181) [jboss-ejb-client-1.0.24.Final-redhat-1.jar:1.0.24.Final-redhat-1]
	at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:144) [jboss-ejb-client-1.0.24.Final-redhat-1.jar:1.0.24.Final-redhat-1]
	at com.sun.proxy.$Proxy16.getPropertyContent(Unknown Source)
	...
Caused by: java.util.concurrent.TimeoutException: No invocation response received in 2000 milliseconds
	at org.jboss.ejb.client.EJBClientInvocationContext$InvocationTimeoutResultProducer.getResult(EJBClientInvocationContext.java:698) [jboss-ejb-client-1.0.24.Final-redhat-1.jar:1.0.24.Final-redhat-1]
	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:272) [jboss-ejb-client-1.0.24.Final-redhat-1.jar:1.0.24.Final-redhat-1]
	...

Resolution

Such network issues are not detected very fast.

The READ and WRITE_TIMEOUT does not help as this xnio settings are passive for performance reasons.
The heartbeat forces Remoting to write to the socket periodically, which in turn forces a timeout check.

To enable a faster failure the remoting protocol is able to send heartbeats to ensure the connection is working.
The time must be set per connection within your configuration, but consider a fast heartbeat will burden your client, server and the network.

Example for a standalone client (jboss-ejb-client.properties)

remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED=false
remote.connections=default

remote.connection.default.host=aHost
remote.connection.default.port=8080
remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false
remote.connection.default.connect.options.org.jboss.remoting3.RemotingOptions.HEARTBEAT_INTERVAL=60000

Note that the HEARTBEAT_INTERVALL need to be set for each connection and the cluster configuration as well.

Example for a server

profile configuration for remote-outbound-connection
<remote-outbound-connection name="remote-ejb-connection" outbound-socket-binding-ref="remote-ejb" username="user" security-realm="ejb-security-realm">
  <properties>
    <property name="SASL_POLICY_NOANONYMOUS" value="false"/>
    <property name="SSL_ENABLED" value="false"/>
    <property name="org.jboss.remoting3.RemotingOptions.HEARTBEAT_INTERVAL" value="60000"/>
  </properties>
</remote-outbound-connection>
jboss-ejb-client.xml

A change here is only necessary if the target is a cluster to set the HEATBEAT_INTERVALL for the clustered connections as well.

<jboss-ejb-client    ...>
    <ejb-receivers>
      <remoting-ejb-receiver outbound-connection-ref="remote-ejb-connection"></remoting>
    </ejb-receivers>
  <client-context>
      <cluster name="ejb" security-realm="ejb-security-realm">
        <connection-creation-options>
          <property name="org.xnio.Options.SSL_STARTTLS" value="true" />
          <property name="org.xnio.Options.SASL_POLICY_NOANONYMOUS" value="false" />
          <property name="org.jboss.remoting3.RemotingOptions.HEARTBEAT_INTERVAL" value="60000" />
        </connection-creation-options>
      </cluster>
    </clusters>
  </client-context>
</jboss-ejb-client>

Related Article

General configuration and further articles can be found in this article:
How to configure an EJB client in JBoss EAP 6

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.