EJB lookup fails when the DNS changes using remote-outbound-connection on JBoss EAP 6

Environment

• Red Hat JBoss Enterprise Application Platform (EAP)
  • 6.x

Issue

• There is two JBoss EAP servers communicating with each other through remote EJB3, server-A and server-B.
  The server-A calls the remote EJB using remote-outbound-connection with the following configuration:

<outbound-connections>
                    <remote-outbound-connection name="ejb-outbound-connection" outbound-socket-binding-ref="serverA2serverB-socket" username="srv2srv" security-realm="ejb-security-realm">
                        <properties>
                            <property name="SSL_ENABLED" value="false"/>
                        </properties>
                    </remote-outbound-connection>
...
<outbound-socket-binding name="serverA2serverB-socket">
                <remote-destination host="server-B" port="4447"/>
            </outbound-socket-binding>

After a outage in the server-B we had to change the DNS entry to point to the backup server. This change should be transparent to the final user but it is not happening, the JBoss should be restarted to the new configuration take effect.

• There is the following parameter configured: networkaddress.cache.ttl=30 in t he $JAVA_HOME/lib/security/java.security. configuration file.

Resolution

Apply JBoss EAP 6.4 Cumulative Patch (CP) 6 or later

Workaround: restart the client server

Root Cause

There is a Bug in the remoting and network subsystems that is caching the IP address, in another words, once the remote connection is created the new IP address will be resolved again only after a reboot.

This content is not included.EAP 6.4.z - https://bugzilla.redhat.com/show_bug.cgi?id=1248156
Content from issues.jboss.org is not included.EAP 7.0.z - https://issues.jboss.org/browse/JBEAP-1960
Content from issues.jboss.org is not included.Wildfly - https://issues.jboss.org/browse/WFCORE-851

The issue is in these two classes, removing these caching resolves the issue.
The methods are responsible to cache the address:

remoting/subsystem/src/main/java/org/jboss/as/remoting/RemoteOutboundConnectionService.java
     private synchronized URI getConnectionURI() throws IOException, URISyntaxException {
        if (this.connectionURI != null) {
            return this.connectionURI;
        }

network/src/main/java/org/jboss/as/network/OutboundSocketBinding.java
     public synchronized InetAddress getResolvedDestinationAddress() throws UnknownHostException {
        if (this.resolvedDestinationAddress != null) {
            return this.resolvedDestinationAddress;
        }

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.