JBoss Messaging clustering problems after a node leaves the cluster

Solution Unverified - Updated 7 Aug 2024

Environment

JBoss Enterprise Application Platform (EAP)
- 4.3
- 5
JBoss Service Oriented Architecture Platform (SOA-P)
- 5

Issue

Clustering issues isolated to the JBoss Messaging JBM-CTRL JGroups channel after a member leaves the cluster

Resolution

For EAP 4.3, update to EAP 4.3 CP10 and install This content is not included.a one-off patch for this issue
For EAP 5, upgrade to JBoss EAP 5.1.2 or later.
For SOA-P, upgrade to SOA-P 5.2.
There are also some one-off patches available for some earlier releases.

Root Cause

Content from issues.jboss.org is not included.https://issues.jboss.org/browse/JBMESSAGING-1893
JBoss Messaging performs long-running tasks in the JGroups thread that updates the cluster view.
During this period, the JBoss Messaging cluster channel cannot process any other messages.

Diagnostic Steps

Are the clustering issues isolated to only the JBM-CTRL JGroups channel ?
Do the issues indicate a node is not responding ?
Does the issue happen during the time between the following logs on one node ?

INFO  [org.jboss.messaging.core.impl.postoffice.MessagingPostOffice] JBoss Messaging is failing over for failed node 3. If there are many messages to reload this may take some time...
INFO  [org.jboss.messaging.core.impl.postoffice.MessagingPostOffice] JBoss Messaging failover completed

Another possible manifestation of this issue is

“WARN  [org.jgroups.protocols.pbcast.GMS] (main) join(<ip_1>:port) sent to <ip_2>:port timed out (after 3000 ms),

SBR

Product(s)

Category

Troubleshoot

Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.