Colocated Live-Backup HornetQ setup in EAP 6 with shared store

Solution Verified - Updated 6 Aug 2024

Environment

Red Hat JBoss Enterprise Application Platform (EAP)
- 6.x
  - Shared store

Issue

How to configure Colocated Live-Backup HornetQ setup in EAP 6?

Resolution

In order to configure Colocated Live-Backup HornetQ servers in EAP 6, you need to configure 2 HornetQ subsystems in each standalone configuration. One HornetQ subsystem will act as live node configuration and the other subsystem will act as the backup node configuration for second server. Please find the configuration below:

Note

Before following settings you need to remove default settings in <hornetq-server>.
This content is not included.GFS2 on a SAN or NFSv4 are the supported configuration for shared stored
In standalone-full-ha.xml, the console log is disabled by default. If necessary, enable the console log by CLI like this.

/subsystem=logging/console-handler=CONSOLE:add(level=INFO,formatter="%K{level}%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n")
/subsystem=logging/root-logger=ROOT:add-handler(name="CONSOLE")

Server 1 configuration

In the standalone-full-ha.xml define 2 paths, One for live and other for backup. This is where the paging, bindings and journal data will be stored

    <paths>
        <path name="jboss.messaging.data.dir1" path="/NotBackedUp/Temp1"/>       
        <path name="jboss.messaging.data.dir2" path="/NotBackedUp/Temp2"/>        
    </paths>

Server 1: live server config

            <hornetq-server name="live1">
             <cluster-password>pass</cluster-password>
                <shared-store>true</shared-store>
                <paging-directory path="paging" relative-to="jboss.messaging.data.dir1"/>
		<bindings-directory path="bindings" relative-to="jboss.messaging.data.dir1"/>
		<journal-directory path="journal" relative-to="jboss.messaging.data.dir1"/>
		<large-messages-directory path="largemessages" relative-to="jboss.messaging.data.dir1"/>
             <connectors>
	      <connector name="netty">
		 <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
		 <param key="host"  value="localhost"/>
		 <param key="port"  value="5445"/>
	      </connector>
                   <in-vm-connector name="in-vm" server-id="0"/>
                </connectors>
                <acceptors>
		      <acceptor name="netty">
			 <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
			 <param key="host"  value="localhost"/>
			 <param key="port"  value="5445"/>
		      </acceptor>
                    <in-vm-acceptor name="in-vm" server-id="0"/>
                </acceptors>
                <jms-connection-factories>
                    <connection-factory name="InVmConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="in-vm"/>
                        </connectors>
                        <entries>
                            <entry name="java:/ConnectionFactory"/>
                        </entries>
                    </connection-factory>
                    <connection-factory name="RemoteConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="netty"/>
                        </connectors>
                        <entries>
                            <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>
                        </entries>
                        <ha>true</ha>
                        <block-on-acknowledge>true</block-on-acknowledge>
                        <retry-interval>1000</retry-interval>
                        <retry-interval-multiplier>1.0</retry-interval-multiplier>
                        <reconnect-attempts>-1</reconnect-attempts>
                    </connection-factory>
                    <pooled-connection-factory name="hornetq-ra">
                        <transaction mode="xa"/>
                        <connectors>
                            <connector-ref connector-name="in-vm"/>
                        </connectors>
                        <entries>
                            <entry name="java:/JmsXA"/>
                        </entries>
                    </pooled-connection-factory>
                </jms-connection-factories>
                <jms-destinations>
                    <jms-queue name="TestQ1">
                        <entry name="java:jboss/exported/TestQ1"/>
                        <durable>true</durable>
                    </jms-queue>
                </jms-destinations>
            </hornetq-server>

Server 1 backup server config

          <hornetq-server  name="backup2">
             <cluster-password>pass</cluster-password>
                <shared-store>true</shared-store>
		<backup>true</backup>
                <journal-file-size>102400</journal-file-size>
                <journal-min-files>2</journal-min-files>
                <paging-directory path="paging" relative-to="jboss.messaging.data.dir2"/>
                <bindings-directory path="bindings" relative-to="jboss.messaging.data.dir2"/>
                <journal-directory path="journal" relative-to="jboss.messaging.data.dir2"/>
                <large-messages-directory path="largemessages" relative-to="jboss.messaging.data.dir2"/>

                <connectors>
	      <connector name="netty_remote">
		 <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
		 <param key="host"  value="localhost"/>
		 <param key="port"  value="5545"/>
	      </connector>

                    <in-vm-connector name="in-vm_local" server-id="1"/>
                </connectors>

                <acceptors>
	      <acceptor name="netty_remote">
		 <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
		 <param key="host"  value="localhost"/>
		 <param key="port"  value="5545"/>
	      </acceptor>
                    <in-vm-acceptor name="in-vm_local" server-id="1"/>
                </acceptors>
             <!--jms-connection-factories>
                    <connection-factory name="InVmConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="in-vm_local"/>
                        </connectors>
                        <entries>
                            <entry name="java:/ConnectionFactory"/>
                        </entries>
                    </connection-factory>
                    <connection-factory name="RemoteConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="netty_remote"/>
                        </connectors>
                        <entries>
                            <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>
                        </entries>
                    </connection-factory>
                    <connection-factory name="MyConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="netty_remote"/>
                        </connectors>
                        <entries>
                            <entry name="JNDI_MyConnectionFactory"/>
                        </entries>
                    </connection-factory>
                </jms-connection-factories-->
           </hornetq-server>

Server 2 configuration

In the standalone-full-ha.xml define 2 paths: One for live and other for backup. This is where the paging, bindings and journal data will be stored

    <paths>
        <path name="jboss.messaging.data.dir1" path="/NotBackedUp/Temp2"/>       
        <path name="jboss.messaging.data.dir2" path="/NotBackedUp/Temp1"/>        
    </paths>

Server2: live server config

           <hornetq-server name="live2">
             <cluster-password>pass</cluster-password>
                <shared-store>true</shared-store>
                <paging-directory path="paging" relative-to="jboss.messaging.data.dir1"/>
		<bindings-directory path="bindings" relative-to="jboss.messaging.data.dir1"/>
		<journal-directory path="journal" relative-to="jboss.messaging.data.dir1"/>
		<large-messages-directory path="largemessages" relative-to="jboss.messaging.data.dir1"/>
	      <connectors>
	      <connector name="netty">
		 <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
		 <param key="host"  value="localhost"/>
		 <param key="port"  value="5545"/>
	      </connector>
                    <in-vm-connector name="in-vm" server-id="1"/>
                </connectors>
                <acceptors>
		      <acceptor name="netty">
			 <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
			 <param key="host"  value="localhost"/>
			 <param key="port"  value="5545"/>
		      </acceptor>
                    <in-vm-acceptor name="in-vm" server-id="1"/>
                </acceptors>
                <jms-connection-factories>
                    <connection-factory name="InVmConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="in-vm"/>
                        </connectors>
                        <entries>
                            <entry name="java:/ConnectionFactory"/>
                        </entries>
                    </connection-factory>
                    <connection-factory name="RemoteConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="netty"/>
                        </connectors>
                        <entries>
                            <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>
                        </entries>
                        <ha>true</ha>
                        <block-on-acknowledge>true</block-on-acknowledge>
                        <retry-interval>1000</retry-interval>
                        <retry-interval-multiplier>1.0</retry-interval-multiplier>
                        <reconnect-attempts>-1</reconnect-attempts>
                    </connection-factory>
                    <pooled-connection-factory name="hornetq-ra">
                        <transaction mode="xa"/>
                        <connectors>
                            <connector-ref connector-name="in-vm"/>
                        </connectors>
                        <entries>
                            <entry name="java:/JmsXA"/>
                        </entries>
                    </pooled-connection-factory>
                </jms-connection-factories>
                <jms-destinations>
                    <jms-queue name="TestQ1">
                        <entry name="java:jboss/exported/TestQ1"/>
                        <durable>true</durable>
                    </jms-queue>
                </jms-destinations>
            </hornetq-server>

Server 2 backup server config

          <hornetq-server name="backup1">
             <cluster-password>pass</cluster-password>
                <shared-store>true</shared-store>
		<backup>true</backup>
                <paging-directory path="paging" relative-to="jboss.messaging.data.dir2"/>
		<bindings-directory path="bindings" relative-to="jboss.messaging.data.dir2"/>
		<journal-directory path="journal" relative-to="jboss.messaging.data.dir2"/>
		<large-messages-directory path="largemessages" relative-to="jboss.messaging.data.dir2"/>

             <connectors>
	      <connector name="netty_remote">
		 <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
		 <param key="host"  value="localhost"/>
		 <param key="port"  value="5445"/>
	      </connector>
                    <in-vm-connector name="in-vm" server-id="0"/>
            </connectors>
                <acceptors>
		      <acceptor name="netty_remote">
			 <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
			 <param key="host"  value="localhost"/>
			 <param key="port"  value="5445"/>
		      </acceptor>
                    <in-vm-acceptor name="in-vm" server-id="0"/>
                </acceptors>

                <!--jms-connection-factories>
                    <connection-factory name="InVmConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="in-vm"/>
                        </connectors>
                        <entries>
                            <entry name="java:/ConnectionFactory"/>
                        </entries>
                    </connection-factory>
                    <connection-factory name="RemoteConnectionFactory">
                        <connectors>
                            <connector-ref connector-name="netty"/>
                        </connectors>
                        <entries>
                            <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>
                        </entries>
                        <ha>true</ha>
                        <block-on-acknowledge>true</block-on-acknowledge>
                        <retry-interval>1000</retry-interval>
                        <retry-interval-multiplier>1.0</retry-interval-multiplier>
                        <reconnect-attempts>-1</reconnect-attempts>
                    </connection-factory>
                    
                </jms-connection-factories-->
     </hornetq-server>

Add if failover should get trigger even when server is killed using Ctrl+C.

CLI script to add a back up server

Please note that the "backup-group-name" would have to be changed on all servers for it to work properly.

batch


/subsystem=messaging/hornetq-server=default:write-attribute(name=shared-store,value=false)
/subsystem=messaging/hornetq-server=default:write-attribute(name=failover-on-shutdown,value=true)
/subsystem=messaging/hornetq-server=default:write-attribute(name=allow-failback,value=true)
/subsystem=messaging/hornetq-server=default:write-attribute(name=journal-type,value=NIO)
/subsystem=messaging/hornetq-server=default:write-attribute(name=journal-file-size,value=102400)
/subsystem=messaging/hornetq-server=default:write-attribute(name=journal-min-files,value=2)
/subsystem=messaging/hornetq-server=default:write-attribute(name=backup-group-name,value=mars-one)


/subsystem=messaging/hornetq-server=backup-server:add(backup=true)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=persistence-enabled,value=true)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=cluster-password, value="${jboss.messaging.cluster.password:CHANGE ME!!}"}
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=shared-store,value=false)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=failover-on-shutdown,value=true)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=allow-failback,value=true)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=journal-type,value=NIO)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=journal-file-size,value=102400)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=journal-min-files,value=2)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=check-for-live-server,value=true)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=check-for-live-server,value=true)
/subsystem=messaging/hornetq-server=backup-server:write-attribute(name=backup-group-name,value=mars-two)
/subsystem=messaging/hornetq-server=backup-server/path=paging-directory:add(path=messaginglargemessages2)
/subsystem=messaging/hornetq-server=backup-server/path=large-messages-directory:add(path=messaginglargemessages2)               
/subsystem=messaging/hornetq-server=backup-server/path=journal-directory:add(path=messagingjournal2)
/subsystem=messaging/hornetq-server=backup-server/path=bindings-directory:add(path=messagingbindings2)

/socket-binding-group=standard-sockets/socket-binding=messaging-backup:add(port=5446)

/subsystem=messaging/hornetq-server=backup-server/in-vm-connector=in-vm:add(server-id=1)
/subsystem=messaging/hornetq-server=backup-server/in-vm-acceptor=in-vm:add(server-id=1)
/subsystem=messaging/hornetq-server=backup-server/remote-connector=netty:add(socket-binding=messaging-backup)
/subsystem=messaging/hornetq-server=backup-server/remote-acceptor=netty:add(socket-binding=messaging-backup)


/subsystem=messaging/hornetq-server=backup-server/broadcast-group=bg-group1:add(socket-binding=messaging-group,broadcast-period=5000,connectors=[netty])
/subsystem=messaging/hornetq-server=backup-server/discovery-group=dg-group1:add(socket-binding=messaging-group,refresh-timeout=10000)
/subsystem=messaging/hornetq-server=backup-server/cluster-connection=my-cluster:add(connector-ref=netty,discovery-group-name=dg-group1,cluster-connection-address=jms)

reload
run-batch

SBR

Messaging & Remoting

Product(s)

Red Hat JBoss Enterprise Application Platform

Components

jbossas

Category

Configure

Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.