Red Hat Enterprise Linux Server hangs with OOM_KILLER messages in /var/log/messages

Environment

• Red Hat Enterprise Linux 6
• Red Hat Enterprise Linux 5
• Red Hat Enterprise Linux 4

Issue

• System hung without console access
• The kernel invoked the oom-killer function, and it killed a random process on the system
• Messages are printed in log files, such as /var/log/messages, which include the process that invoked oom-killer and a call trace of the oom-killer. For example:

kernel: httpd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
kernel: 
kernel: Call Trace:
[<ffffffff800c3a6a>] out_of_memory+0x8e/0x2f5
[<ffffffff8000f2eb>] __alloc_pages+0x245/0x2ce
[<ffffffff80012a62>] __do_page_cache_readahead+0x95/0x1d9
[<ffffffff80063a04>] __wait_on_bit_lock+0x5b/0x66
[<ffffffff88147d4d>] :dm_mod:dm_any_congested+0x38/0x3f
[<ffffffff80013401>] filemap_nopage+0x148/0x322
[<ffffffff80008863>] __handle_mm_fault+0x1f8/0xe5c
[<ffffffff80066b9a>] do_page_fault+0x4cb/0x830
[<ffffffff80063097>] thread_return+0x62/0xfe
[<ffffffff8005dde9>] error_exit+0x0/0x84

Resolution

• The oom_kill.c is the file in mm tree of kernel which is responsible to terminate one or more processes in order to free up enough memory so that the system remains operational.

• The badness() function is responsible to elect which processes will be terminated using a scoring heuristic, and the memory size of the process is the basis for the badness().

• The following algorithm is used to account the scoring system of badness():

*
* 1) we lose the minimum amount of work done
* 2) we recover a large amount of memory
* 3) we don't kill anything innocent of eating tons of memory
* 4) we want to kill the minimum amount of processes (one)
* 5) we try to kill the process the user expects us to kill, this
 *    algorithm has been meticulously tuned to meet the principle
 *    of least surprise ... (be careful when you change it)

*

• Processes with PF_SWAPOFF flag will be killed first

/*
         * swapoff can easily use up all memory, so kill those first.
         */

             if (p->flags & PF_SWAPOFF)
                                 return ULONG_MAX; 

• Processes that fork a lot of child processes have more to be killed

        list_for_each_entry(child, &p->children, sibling) {
                  task_lock(child);
                  if (child->mm != mm && child->mm)
                       points += child->mm->total_vm/2 + 1;
                  task_unlock(child);
             }

• Niced processes

        if (task_nice(p) > 0)
                  points *= 2;

• Avoid killing superuser processes

        if (has_capability_noaudit(p, CAP_SYS_ADMIN) ||
                 has_capability_noaudit(p, CAP_SYS_RESOURCE))
                  points /= 4;

• Avoid killing a process with direct hardware access

        if (has_capability_noaudit(p, CAP_SYS_RAWIO))
                  points /= 4;

• It will also kill any process sharing the same memory struct as the selected process.

Root Cause

• The out_of_memory function is called when the system memory (including swap) has been fully allocated to a point where regular system activities cannot be performed until some of that memory is freed.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.