Red Hat Fuse log:tail cause a java.lang.OutOfMemoryError: Java heap space

Solution Verified - Updated

Environment

  • Red Hat Fuse 6.x
  • Camel

Issue

  • I am running a Camel route which logs a large body of text ~1MB in size, after using the log:tail command the container encounters the following error:
2.12.0.redhat-610379 | Error processing exchange. This exception will be ignored, to let the timer be able to trigger again.
java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-19-thread-1" java.lang.OutOfMemoryError: Java heap space

Resolution

  • It is recommended to avoid use of log:tail from karaf shell. Use 'tail' command from linux shell to observe logs at run time.
  • This issue can be resolved by either increasing the max heap size of the JVM or decreasing the log event cache.

The log event cache size is configurable from the $JBOSS_FUSE/etc/org.apache.karaf.log.cfg, change the property "size" from 500 to 100(to whatever reasonable with your log text), then the OOM should not occur.

Root Cause

This is caused by the large log text which can't work with the default heap memory size in JBoss FUSE 6.1.
So by default, Karaf will maintain a log event cache which saves the latest 500 log events (this cache will be used by log:display, log:display-exception command), so we can get 500 PaxLoggingEvent instances, each instance needs memory about twice the size as the log text(1M in this case), so for the default 500 log event cache, with this testcase, we need 1G java heap size, and this exceeds the default JBoss FUSE 6.1 configuration, that's why the OOM occurred. This is not a memory leak, it's just exceeding the default heap size.

SBR
Product(s)
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.