How to enable and view ejb invocation & pool statistics in JBoss EAP 8 / 7 / 6

Solution Verified - Updated

Environment

  • Red Hat JBoss Enterprise Application Platform (EAP)
    • 8.x
    • 7.x
    • 6.x

Issue

  • How to enable ejb statistics in JBoss EAP 7/ 6?
  • Is there a statistic for EJB invocations like this was in former EAP provided by jmx or admin console?
  • The statistic can be seen in the CLI for the deployment, but other values as pool-* are '0'
  • Where can I find invocation statistics for my EJB's?
  • EJB metrics execution-time and invocations always 0 in jboss-cli
  • Number of active sessions (stateful session bean)
  • EJB invocation count in MBean and JBoss CLI is not increasing.
  • There is a requirement to use MBean/JBoss CLI to capture how many times an EJB has been invoked. Attribute "invocations" in EJB's backing MBean should be able to provide this count. But despite numerous invocations , count does not increase. How to get these statistics in EAP 6?
  • How to retrieve the JNDIView in EAP 6?
  • How to get a count of current, total, passivated, removed EJB instances (SFSB, SLSB)?
  • How to Monitor EJB bean pool and which pool is used
  • How can I see which ejb pool is used for the ejb?
  • Can I get EJB invocation / pool stats jmx ?
  • I have doubt about ejb monitoring .I haven't enabled the statistics parameter in ejb subsysystem like below but still I can see ejb statistics through JConsole.
<statistics enabled="false"/>

Resolution

    <subsystem xmlns="urn:jboss:domain:ejb3:1.3">
       ...
       <statistics enabled="true"/>
    </subsystem>

or it can also be done via CLI

bin/jboss-cli.sh --connect
[standalone@localhost:9999 /] /subsystem=ejb3:write-attribute(name=enable-statistics, value=true)

# or in domain mode i.e. to enable it for the default profile
[domain@localhost:9999 /] /profile=default/subsystem=ejb3:write-attribute(name=enable-statistics, value=true)

The EJB stats can be queried using the JBoss CLI:

  • For a specific EJB 
/deployment=helloWorld.ear/subdeployment=helloWorld-ejb.jar/subsystem=ejb3/stateless-session-bean=HelloBean:read-resource(include-runtime=true)
  • For a specific deployment
/deployment=helloWorld.ear/subdeployment=helloWorld-ejb.jar/subsystem=ejb3:read-resource(include-runtime=true, recursive=true)
  • For all ear deployments (or those with sub deployments)
/deployment=*/subdeployment=*/subsystem=ejb3:read-resource(include-runtime=true, recursive=true) > ejb-stats.log
  • For all non deployments (those with out sub deployments)
/deployment=*/subsystem=ejb3:read-resource(include-runtime=true, recursive=true) > ejb-stats.log
  • For a all deployments, redirecting the output to a file
/deployment=*/subdeployment=*/subsystem=ejb3:read-resource(include-runtime=true, recursive=true) > ejb-stats.log

Standalone Mode

  • The EJB statistics are at this location for a stateless EJB example of helloWorld.ear/helloWorld-ejb.jar/HelloBean.
/deployment=helloWorld.ear/subdeployment=helloWorld-ejb.jar/subsystem=ejb3/stateless-session-bean=HelloBean:read-resource(include-runtime=true)
  • These bean types are also available under the subsystem=ejb3: entity-bean, message-driven-bean, singleton-bean, stateful-session-bean, stateless-session-bean or you can do a recursive to see all of the ejbs stats
/deployment=helloWorld.ear/subdeployment=helloWorld-ejb.jar/subsystem=ejb3:read-resource(include-runtime=true, recursive=true)

Domain Mode

  • Make sure in domain mode that you add it to the correct profile.

  • Then you run read-resource on the ejb or deployment you want this examples shows for my example domain with host named master, and server group named server-one, I have a deployment helloWorld.ear/helloWorld-ejb.jar/HelloBean.

  • To view the statistics on HelloBean on host: master, server-one, the command would look like this:

/host=master/server=server-one/deployment=helloWorld.ear/subdeployment=helloWorld-ejb.jar/subsystem=ejb3/stateless-session-bean=HelloBean:read-resource(include-runtime=true)

  • Recursive can be specified just as in the standalone mode.

  • Due to Content from issues.jboss.org is not included.AS7-6738 the time which ejb-invocations are blocked 'wait-time' is not calculated in EAP6.0.

  • According to This content is not included.Bug 911117, the following metrics are missing in EAP 6 EJB3 subsystem from deployment as to compared to EAP 5 exposed metrics :

    ####For Stateless Session Bean(SLSB's)

    • Method Invocation Time - The minimum, maximum, and average invocation times for each of the methods exposed by this EJB

    ####For Stateful Session Bean(SFSB's)

    • Method Invocation Time - The minimum, maximum, and average invocation times for each of the methods exposed by this EJB
    • Cache Size
    • Passivated Count
    • Total Size

  • However, this issue has been resolved in EAP 6.1.1. So a possible workaround could be to upgrade to EAP 6.1.1 or later.

Examples

Statistics not enabled

When statistics are not enabled, the output will appear as shown below, note that the pool-available-count, pool-create-count, pool-current-size, and pool-max-size have a value, but the execution-time, invocations, methods and others do not have a value because statistics has not been enabled.

[standalone@localhost:9999 /] /deployment=helloWorld.ear/subdeployment=helloWorld-ejb.jar/subsystem=ejb3:read-resource(include-runtime=true, recursive=true)
{
    "outcome" => "success",
    "result" => {
        "entity-bean" => undefined,
        "message-driven-bean" => undefined,
        "singleton-bean" => undefined,
        "stateful-session-bean" => undefined,
        "stateless-session-bean" => {"HelloBean" => {
            "component-class-name" => "HelloBean",
            "declared-roles" => [],
            "execution-time" => 0L,
            "invocations" => 0L,
            "methods" => {},
            "peak-concurrent-invocations" => 0L,
            "pool-available-count" => 19,
            "pool-create-count" => 1,
            "pool-current-size" => 1,
            "pool-max-size" => 20,
            "pool-name" => "slsb-strict-max-pool",
            "pool-remove-count" => 0,
            "run-as-role" => undefined,
            "security-domain" => "other",
            "timers" => [],
            "wait-time" => 0L
        }}
    }
}

Statistics enabled

Now, the invocations, methods, peak-concurrent-invocations are being tracked since statistics are enabled.

Stateless Session Bean Example:

/deployment=helloWorld.ear/subdeployment=helloWorld-ejb.jar/subsystem=ejb3/stateless-session-bean=HelloBean:read-resource(include-runtime=true,recursive=true)
{
    "outcome" => "success",
    "result" => {
        "component-class-name" => "HelloBean",
        "declared-roles" => [],
        "execution-time" => 1L,
        "invocations" => 4L,
        "methods" => {"hello" => {
            "execution-time" => 1L,
            "invocations" => 4L,
            "wait-time" => 8L
        }},
        "peak-concurrent-invocations" => 1L,
        "pool-available-count" => 20,
        "pool-create-count" => 1,
        "pool-current-size" => 1,
        "pool-max-size" => 20,
        "pool-name" => "slsb-strict-max-pool",
        "pool-remove-count" => 0,
        "run-as-role" => undefined,
        "security-domain" => "other",
        "timers" => [],
        "wait-time" => 8L
    }
}

Message Driven Bean Example:

/deployment= application2/subdeployment=consumer-ejb.jar/subsystem=ejb3:read-resource(include-runtime=true,recursive=true)

    {
        "entity-bean" => undefined,
        "message-driven-bean" => {"MessageConsumer2" => {
            "component-class-name" => "MessageConsumer2",
            "declared-roles" => [],
            "delivery-active" => true,
            "execution-time" => 46L,
            "invocations" => 152L,
            "methods" => {"onMessage" => {
                "execution-time" => 46L,
                "invocations" => 152L,
                "wait-time" => 44L
            }},
            "peak-concurrent-invocations" => 1L,
            "pool-available-count" => 20,
            "pool-create-count" => 1,
            "pool-current-size" => 1,
            "pool-max-size" => 20,
            "pool-name" => "mdb-strict-max-pool",
            "pool-remove-count" => 0,
            "run-as-role" => undefined,
            "security-domain" => "other",
            "timers" => [],
            "wait-time" => 44L,
            "service" => undefined
        }},
        "singleton-bean" => undefined,
        "stateful-session-bean" => undefined,
        "stateless-session-bean" => undefined
    }

stateless statistics:

method => { 
  execution-time: the execution time of the given method for this ejb in ms
  invocations: the number of times the given method has been invoked on this ejb
  wait-time: the wait time for the given method in ms
}

peak-concurrent-invocations: the maximum number of concurrent invocations at any given time
pool-available-count:
pool-create-count: how many instances have been created for the pool
pool-current-size: the current size of the instance pool
pool-max-size: the maximum size that the instance pool can reach
pool-name: the pool configuration name
pool-remove-count: the number of instances removed from the pool
execution-time: total execution-time (sum of all execution-time from all of the methods) in ms (milliseconds)
wait-time: total wait time (sum of the wait-time from all of the methods) in ms (milliseconds) , if the EJB has an instance pool configured then this is how long it waited for a free instance, if the instance pool is disabled, then this time is essentially 0 as it is the time creating the EJB instance
invocations: the total number of invocations made on this ejb

Notes:

  • The execution-time and wait-time should be divided by the invocations to get the averages, since these numbers are totals, not individual requests
    • average execution-time = execution-time by the invocations
    • average wait-time = wait-time by the invocations
  • If the average wait-time is high for a particular EJB, then you might want to increase the instance pool size for that particular EJB to allow more concurrent invocations
    • Also removing the default instance pool for stateless EJBs would allow any available thread to create an instance
  • If the average execution-time is high for a particular EJB / EJB method, then you can look at what the EJB method is doing, resources it uses (Database connections), etc and see if anything needs to be tuned to provide better performance.
  • The peak-concurrent-invocations
    • Is useful to know how many concurrent requests are coming in for a particular EJB to know if it might need to increase the instance pool to allow more concurrent invocations
    • Summing all peak-concurrent-invocations for all EJBs would be the potential maximum number of concurrent invocations across all EJBs, comparing that to the thread pool size used for the remote EJBs would show if the thread pool max is being reached or not.

stateful statistics:

methods => {
  execution-time: the execution time of the given method for this ejb in ms
  invocations: the number of times the given method has been invoked on this ejb
  wait-time: the wait time for the given method in ms
}

cache-size: the number of ejbs in the cache
execution-time: total execution-time (sum of all execution-time from all of the methods) in ms (milliseconds)
invocations: the total number of invocations made on this ejb
passivated-count: the number of SFSB instances that are passivated
peak-concurrent-invocations: the maximum number of concurrent invocations at any given time
wait-time: total wait time (sum of the wait-time from all of the methods) in ms (milliseconds)

JMX MBean (jconsole) is also visible in jconsole

For example, an EJB named HelloBean in helloWorld.ear/helloWorld-ejb.jar would have an ObjectName of:

jboss.as:deployment=helloWorld.ear,subdeployment=helloWorld-ejb.jar,subsystem=ejb3,stateless-session-bean=HelloBean

How to set an EJB's Instance Pool class via annotation and xml in JBoss EAP 6
@Pool vs maxSession for MDB in JBoss EAP 6

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.