Syscall to futex causes 100% cpu utilization by Java application using LockSupport.parkNanos()

Environment

• Red Hat Enterprise Linux (RHEL) 8
• Java

Issue

• We see 100% CPU utilization on Redhat Enterprise Linux 8 (RHEL8) kernel 4.18 as compared to around 25% on RHEL7 kernel 3.10 when executing the same Java method LockSupport.parkNanos() to park threads.
• Using lmax in our java application, we see it consumes higher CPU after moving to RHEL 8 with CPU in threads like below:

   java.lang.Thread.State: TIMED_WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
    at com.lmax.disruptor.SleepingWaitStrategy.applyWaitMethod(SleepingWaitStrategy.java:92)
    at com.lmax.disruptor.SleepingWaitStrategy.waitFor(SleepingWaitStrategy.java:65)
    at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
    at com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:159)
    at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Resolution

• Do not use java.util.concurrent.locks.LockSupport.parkNanos() for times less thantimerslack_ns (default 50000 nanoseconds).
• For smaller intervals, likely a different approach is needed (e.g. kernel-rt and not Java code). Or, if the intent is to yield, call Thread.yield(), which uses sched_yield().
• If using lmax, update it to lmax 3.3.9+ to address its Content from github.com is not included.CPU issues. But note there are still some Content from github.com is not included.outstanding issues to be addressed in some other points of lmax like

Root Cause

Calls to java.util.concurrent.locks.LockSupport.parkNanos() for times less thantimerslack_ns (default 50000 nanoseconds) return immediately without blocking in the kernel due to RHEL8's improved timeout implementation that passes an expired timer to the kernel that, instead of sleeping for an extra timerslack_ns before returning, just returns.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.