How to use the LeakDumperManagedConnectionPool facility to detect leaked datasource connections in JBoss EAP 6.4 and 7.x

Solution Verified - Updated

Environment

  • Red Hat JBoss Enterprise Application Platform (EAP)
    • 6.4 and later
    • 7.x

Issue

  • How can the LeakDumperManagedConnectionPool utility be enabled?
  • How can the LeakDumperManagedConnectionPool utility be used to detect connection leaks?
  • Need to detect connection leaks in an older EAP release where CCM debug does not work for non-JTA datasources

Resolution

Limitations

  • The leak dumper utility reports "potential" as well as actual leaks. For instance, if a connection is in use at the instant the leak dumper report is generated, it will be reported as a leak though it might not be leaked (i.e. it might normally be returned to the pool by application code after completion of the in-progress activity). The nature of the leak dumper tool, therefore, involves some ambiguity so use of the cached connection manager (CCM) debug facility is recommended/preferred where possible (note that in some EAP 6 releases and in EAP 7.0, CCM debug could only be used with jta-compatible pools).
  • Note that restart or flush of the pool (rather than shutting down EAP/stopping the JVM) may result in application errors.
    • Any connections that are in-use at the time of pool restart/flush will fail.
    • If the application/3rd party code fails to retrieve datasource pool references via JNDI lookup prior to each getConnection request, failures may occur as restart/flush of the pool invalidates any existing datasource references. Caching of datasource references is an unsupported use case.
  • Use the leak dumper pool for non-JTA-enabled pools (i.e. pools used with legacy code which requires jta="false" be configured for the pool) in JBoss EAP 6.4 and JBoss EAP 7.0.
  • Use the leak dumper pool, with JBoss EAP 6.4 and later, for pools where the cached connection manager has been disabled.
    • Note that disabling the cached connection manager is not recommended for most systems. Use of the cached connection manager (in non-debug mode) is standard configuration in both production and development systems. CCM debug is not the same as the cached connection manager (i.e. the use-ccm="true" configuration will not enable debug facilities unless cached-connection-manager debug is separately enabled).
  • For scenarios other than the two above, the cached connection manager (CCM) debug facility is recommended/preferred rather than the leak dumper pool.

How to configure

  • In EAP 7.x, you can enable it by setting org.jboss.jca.core.connectionmanager.pool.mcp.LeakDumperManagedConnectionPool to the mcp attribute in your datasource configuration. For example:

    /subsystem=datasources/data-source=<your-pool-name>:write-attribute(name=mcp,value=org.jboss.jca.core.connectionmanager.pool.mcp.LeakDumperManagedConnectionPool)

  • In EAP 6.4 or later, you can enable it by the following system properties:

    -Dironjacamar.mcp=org.jboss.jca.core.connectionmanager.pool.mcp.LeakDumperManagedConnectionPool
-Dironjacamar.leaklog=<filename>
    • These may be configured in various ways including the following:
      • Add the properties as JAVA_OPTS in <JBOSS_HOME>/bin/<standalone|domain>.<sh|bat>
      • Add the properties to the <system-properties> section of the <standalone|domain>.xml` as noted in this knowledge solution

How it works

  • A stack trace for all connections still retained by application components will be written to server.log (as ERROR level of org.jboss.jca.core.connectionmanager.pool.mcp.LeakDumperManagedConnectionPool category) in EAP 7.x and the specified file in EAP 6.4 when any of the following occur:
    • EAP is shutdown
    • The relevant datasource pool is restarted
    • The relevant datasource pool is flushed using flush-all-connection-in-pool
      • Note that this action will terminate all connections including those which may have on-going operations initiated by application components

References:

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.