JON Agent JVM consuming a lot of memory due to many bindings set in bindings.xml

Solution Unverified - Updated

Environment

  • Red Hat JBoss Operations Network (JON) 3.0, 3.1
  • Red Hat JBoss Enterprise Application Platform (EAP) 5.1
  • Many binding sets defined in $JBOSS_HOME/server/$PROFILE/conf/bindingservice.beans/META-INF/bindings-jboss-beans.xml

Issue

  • Agent consuming a lot of memory
  • Agent constant OutOfMemory
  • agent running out of memory
  • Additional binding set definitions seem to require significantly more memory

Resolution

Reduce the number of binding sets deployed to the EAP server. See How to configure binding sets and injecting only a single one for details.

Root Cause

The default out-of-the-box mechanism for configuring binding sets results in every configured bean being injected even though the EAP 5.x instance is only using one of the injected beans.
e.g

   <bean name="ServiceBindingManagementObject" 
         class="org.jboss.services.binding.managed.ServiceBindingManagementObject">

      <constructor>
         <parameter>${jboss.service.binding.set:ports-default}</parameter>
         <parameter>
            <set>
               <inject bean="PortsDefaultBindings"/>
               .. .. ..
               <inject bean="Ports03Bindings"/>
            </set>
         </parameter>
         <parameter><inject bean="StandardBindings"/></parameter>

      </constructor>
   </bean>

Diagnostic Steps

  • Capture a heap dump
  • Analyse the heap:

    • Look at the Dominator Tree, down through the larest objects at each level

    • See that the top is the InventoryManager, and down inside the map is the org.rhq.plugins.jbossas5.serviceBinding.ManagerComponent instance

    • Review how many instances of Resource represent a service binding set
      The following OQL query can be used:

        SELECT toString(r.name), toString(r.resourceKey), toString(r.inventoryStatus.name) FROM INSTANCEOF org.rhq.core.domain.resource.Resource r WHERE (toString(r.resourceKey) LIKE "ServiceBindingManager.*")
SBR
Product(s)
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.