Explanation of the JCA JDBC pool statistics for JBoss EAP

Environment

• Red Hat JBoss Enterprise Application Platform (EAP)
  • 7
  • 6
• Java Connector Architecture (JCA)
  • 1.7
  • 1.6
• IronJacamar 1

Issue

• What are the meanings of the data-source statistics reported by the CLI statistics read command?
• How are datasource metrics calculated?

[standalone@localhost:9990 /] /subsystem=datasources/data-source=ExampleDS/statistics=pool:read-resource(include-runtime=true)
{
    "outcome" => "success",
    "result" => {
        "ActiveCount" => 1,
        "AvailableCount" => 20,
        "AverageBlockingTime" => 0L,
        "AverageCreationTime" => 110L,
        "AverageGetTime" => 114L,
        "AveragePoolTime" => 0L,
        "AverageUsageTime" => 0L,
        "BlockingFailureCount" => 0,
        "CreatedCount" => 1,
        "DestroyedCount" => 0,
        "IdleCount" => 1,
        "InUseCount" => 0,
        "MaxCreationTime" => 110L,
        "MaxGetTime" => 114L,
        "MaxPoolTime" => 0L,
        "MaxUsageTime" => 0L,
        "MaxUsedCount" => 1,
        "MaxWaitCount" => 0,
        "MaxWaitTime" => 0L,
        "TimedOut" => 0,
        "TotalBlockingTime" => 0L,
        "TotalCreationTime" => 110L,
        "TotalGetTime" => 114L,
        "TotalPoolTime" => 0L,
        "TotalUsageTime" => 0L,
        "WaitCount" => 0,
        "XACommitAverageTime" => 0L,
        "XACommitCount" => 0L,
        "XACommitMaxTime" => 0L,
        "XACommitTotalTime" => 0L,
        "XAEndAverageTime" => 0L,
        "XAEndCount" => 0L,
        "XAEndMaxTime" => 0L,
        "XAEndTotalTime" => 0L,
        "XAForgetAverageTime" => 0L,
        "XAForgetCount" => 0L,
        "XAForgetMaxTime" => 0L,
        "XAForgetTotalTime" => 0L,
        "XAPrepareAverageTime" => 0L,
        "XAPrepareCount" => 0L,
        "XAPrepareMaxTime" => 0L,
        "XAPrepareTotalTime" => 0L,
        "XARecoverAverageTime" => 0L,
        "XARecoverCount" => 0L,
        "XARecoverMaxTime" => 0L,
        "XARecoverTotalTime" => 0L,
        "XARollbackAverageTime" => 0L,
        "XARollbackCount" => 0L,
        "XARollbackMaxTime" => 0L,
        "XARollbackTotalTime" => 0L,
        "XAStartAverageTime" => 0L,
        "XAStartCount" => 0L,
        "XAStartMaxTime" => 0L,
        "XAStartTotalTime" => 0L,
        "statistics-enabled" => true
    }
}

Note that available statistics will vary with JBoss EAP version.

Resolution

Description of individual/specific pool statistic properties may be found in the release version specific configuration guide for JBoss EAP. The documents for the most recent JBoss EAP 6 and 7 releases are:

• EAP 6 Administration and Configuration guide and
• EAP 7 Configuration Guide

There are two types of statistics:

1. Simple counters (typically suffixed with Count) that reflect the current state or history of the connection pool. Among the commonly evaluated counters (e.g. when analyzing health/performance of the pool) are the following¹:
   • InUseCount - The number of physical connections that are reserved by application components which have executed a getConnection() request but not yet returned the connection to the pool by calling close().
     • These connections may or may not be associated with in-progress SQL command execution but the connections are in a reserved state (owned by the application) and are not available to be returned by getConnection() requests. This count may include connections that have been leaked by application code which fails to call Connection.close() (i.e. to release the connection back to the pool) after use of the connection.
     • A value for this counter which approaches the configured max-pool-size may represent an undersized pool (too small to satisfy concurrent demand during peak load) or incidents of connection leaks.
   • IdleCount - The number of existing connections that are in the idle/free pool (not currently reserved by application components)
     • These connections are not participants in active transactions nor in-progress SQL command executions.
   • MaxUsedCount - The maximum number of connections used at one time - a "high water" mark
     • A high value for this counter may reflect a present or past instant (or timeframe) during which concurrent demand for connections was high due to load or performance issues. When MaxUsedCount is observed to be close to the configured max-pool-size, concurrent load analysis may be warranted or an increase in max-pool-size may be appropriate. Note that this count will differ from InUseCount when statistics are taken if the number of in-use connections has dropped since the MaxUsedCount was last reached.
   • ActiveCount - The number of open/existing connections - whether in a reserved/in-use state or an unreserved/idle state
     • A high value for this counter may correlate to a high number of open connections reported by database server tooling.
   • AvailableCount - The number of connections that can currently be handed out to satisfy getConnection() requests
     • These connections may not yet exist but could be created, if needed, within the limits of the configured max-pool-size so the value is equal to max-pool-size - InUseCount.
   • DestroyedCount - The number of connections destroyed
     • A high number of destroyed connections may suggest a high number of external connection failures (e.g. network or database server initiated connection closure) or may coincide with a high number of TimedOut connections.
   • TimedOut - The number of connections destroyed by the idle remover thread (as configured by idle-timeout-minutes)
2. Time-based statistics (suffixed with Time) that reflect average, maximum or total millisecond times for important connection or pool events among which are the following²:
   • *UsageTime - Time connections have spent in a reserved state (owned by application components) between the initial getConnection() request and the time Connection.close() is called by the application to release the connection back to the pool
     • A high AverageUsageTime indicates long periods during which connections were held by application components and may suggest performance problems that relate either to application activity (e.g. expensive calculations, long waits on external resource while holding a connection, etc.) or long running SQL commands.
   • *CreationTime - Time spent creating new connections to the datasource.
     • A high AverageCreationTime may indicate potential problems with the network or database (i.e. the JDBC driver is encountering long waits during negotiation with the database server while attempting to create new connections).
   • *BlockingTime - Time spent waiting for a connection to be supplied by the pool.
     • A high AverageBlockingTime³ may indicate an undersized pool (too small to satisfy concurrent demand during peak load) or a general performance problem where application components fail to complete operations in a timely manner and return the connections used.

Note that current releases include statistics for XA operations (such as XA commit and prepare). The counter and time-based statistics may be useful for XA datasource pool evaluation/monitoring.

Individual values above can be explicitly retrieved using CLI commands similar to the below:

[standalone@localhost:9999 /] /subsystem=datasources/data-source=ExampleDS/statistics=pool:read-attribute(name=CreatedCount)

See Also

• Enable and Monitor Pool Statistics

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.