Host Controller can not reconnect to Domain Controller after the connection is closed unexpectedly in the JBoss EAP 7

Solution Verified - Updated

Environment

  • Red Hat JBoss Enterprise Application Platform (EAP)
    • 7.1
    • 7.2.0
  • Managed Domain mode with multiple hosts
  • Often after domain controller restart

Issue

  • The connection to Domain Controller closed unexpectedly and the Host Controller can not reconnect with following error:

    Domain Controller host-controller.log:

      WARN  [org.jboss.as.domain.controller] (management I/O-1) WFLYHC0030: Connection to remote host "hostcontroller-01" closed unexpectedly
  INFO  [org.jboss.as.protocol] (management I/O-1) WFLYPRT0057:  cancelled task by interrupting thread Thread[Host Controller Service Threads - 209,5,Host Controller Service Threads]
  INFO  [org.jboss.as.protocol] (management I/O-2) WFLYPRT0057:  cancelled task by interrupting thread Thread[Host Controller Service Threads - 209,5,Host Controller Service Threads]

    Host Controller host-controller.log:

      INFO  [org.jboss.as.host.controller] (Host Controller Service Threads - 377) WFLYHC0150: Trying to reconnect to master host controller.
  ERROR [org.jboss.as.host.controller] (Host Controller Service Threads - 381) WFLYHC0143: Failed to apply domain-wide configuration from master host controller. Operation outcome: failed. Failure description "WFLYCTL0313: Unauthorized to execute operation 'server-set-reload-required' for resource '[]' -- \"WFLYCTL0332: Permission denied\""

  • Slaves no longer connected to console

Resolution

Patch your instance of JBoss EAP 7.1.x to 7.1.6 or Upgrade to JBoss EAP 7.2.1 or later.

At runtime, it can be resolved by forcing a reload of the Host and/or the Domain Controller without restarting the servers to reestablish the connection between them by running the following command on the JBoss EAP CLI:

reload --host=<host-name> --restart-servers=false

Root Cause

This is a known issue Content from issues.jboss.org is not included.JBEAP-15547 in JBoss EAP 7.1.0 and 7.2.0 which was fixed in versions 7.1.6 and 7.2.1.

The host controller is in reload-required state after long GC pause in the managed servers.

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.