JBoss EAP KUBE_PING clustering in OpenShift Container Platform

Solution Verified - Updated

Environment

  • OpenShift Container Platform 4
  • JBoss Enterprise Application Platform 7.4

Issue

  • How to configure JBoss EAP clustering in OpenShift?

Resolution

See main solution EAP 7 image clustering in OCP 4 for details on clustering explanation, EAP template deployment, EAP 7 Operator, DNS_PING vs KUBE_PING.

JBoss EAP image is configured to use KUBE_PING which uses Kubernetes API to discover clustering members. To enable clustering, we need to allow JBoss EAP service account access to the Kubernetes API.

It's recommended that always starts from templates like eap70-basic-s2i. Do not directly use images such as registry.access.redhat.com/jboss-eap-7/eap70-openshift or imagestreams like jboss-eap70-openshift.

The reason is that the KUBE_PING needs various environment variables and named container port definition in DeploymentConfig to form a cluster. They are pre-configured in these templates along with some other useful things such as livenessProbe and readinessProbe. If you use image directly, the required configuration is missing so you need additional lengthy manual configurations and copy them from the templates. This is unnecessary effort.

Required Items

  1. A headless service with the correct pods labels.
  2. A port named ping must be defined. The Red Hat templates default this port to 8888.
  3. The service account needs permissions to view the default namespace and current namespace.
  4. The variables OPENSHIFT_KUBE_PING_NAMESPACE, JGROUPS_PING_PROTOCOL=kubernetes.KUBE_PING, OPENSHIFT_KUBE_PING_LABELS must be set according. They are deploymentconfig - for deploymentConfig deployment, or WildflyServer CR for EAP 7 Operator.

Using template, to allow JBoss EAP service account access to Kubernetes API, you need to add view role to the service account, in this example default.

$ oc project YOUR_PROJECT
$ oc policy add-role-to-user view system:serviceaccount:$(oc project -q):eap-service-account
$ oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default

Then create a JBoss EAP app with templates like eap70-basic-s2i, scale up to 2 replicas.

$ oc new-app eap70-basic-s2i
$ oc scale dc eap-app --replicas=2

In oc logs we can see 2 pods formed a cluster in the following lines:

$ oc logs eap-app-1-8l9fw  | grep 'new cluster view'
01:22:43,070 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-4) ISPN000094: Received new cluster view for channel web: [eap-app-1-dlt3p|1] (2) [eap-app-1-dlt3p, eap-app-1-8l9fw]
01:22:43,081 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000094: Received new cluster view for channel ejb: [eap-app-1-dlt3p|1] (2) [eap-app-1-dlt3p, eap-app-1-8l9fw]
01:22:43,081 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000094: Received new cluster view for channel server: [eap-app-1-dlt3p|1] (2) [eap-app-1-dlt3p, eap-app-1-8l9fw]
01:22:43,084 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-5) ISPN000094: Received new cluster view for channel hibernate: [eap-app-1-dlt3p|1] (2) [eap-app-1-dlt3p, eap-app-1-8l9fw]

This content is not included.EAP 7 Guide Openshift Clustering

Root Cause

See main solution EAP 7 image clustering in OCP 4 for details on clustering explanation, EAP template deployment, EAP 7 Operator, DNS_PING vs KUBE_PING.

Diagnostic Steps

  1. Verify the kube ping label and details:
   KubePING configuration: KubePing{namespace='example', labels='application=example'}

And compare that with the pods' labels, on this case the pod must have the label application=example, otherwise it won't cluster.

  1. User can get inspect for fully accessing deployment/pod labels, or even pod yaml for example the one below doesn't have label application=example:
apiVersion: v1
kind: Pod
metadata:
  annotations:
   ...
  labels:
    app.version: 1.2345-6
    deployment: example
    pod-template-hash: somehash
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.