The output of 'free -m' shows a large memory usage which is not represented in the output of 'top'

Environment

• Red Hat Enterprise Linux (All versions)

Issue

• On a Red Hat Enterprise Linux system, just after reboot, the output of 'free -m' shows 17GB of memory usage, while the output of 'top' shows no processes consuming large amount of memory.

    # free -m
                 total       used       free     shared    buffers     cached
    Mem:         32181      18267      13913          0         42        305
    -/+ buffers/cache:      17920      14261
    Swap:        24575          0      24575
  

Resolution

• The output of 'free -m' in the system is not wrong.
  • The confusion came about while comparing the outputs of 'free -m' and 'top', without considering the memory reserved for HugePages.

Root Cause

• The memory pages  for HugePages is kept apart at boot-time of the system, and cannot be used  for other memory allocations, even if they are free. If the HugePages are not used by any process in the system, the output of 'top' will not show this 'used'  (reserved), but 'un-utilised' part of memory.

Diagnostic Steps

• Get the output of /proc/meminfo

    # cat /proc/meminfo
    MemTotal: 32954140 kB
    MemFree: 14247824 kB
    Buffers: 43172 kB
    Cached: 312868 kB
    SwapCached: 0 kB
    Active: 159084 kB
    Inactive: 217392 kB
    HighTotal: 0 kB
    HighFree: 0 kB
    LowTotal: 32954140 kB
    LowFree: 14247824 kB
    SwapTotal: 25165816 kB
    SwapFree: 25165816 kB
    Dirty: 8 kB
    Writeback: 0 kB
    Mapped: 26932 kB
    Slab: 51960 kB
    CommitLimit: 32529284 kB
    Committed_AS: 36568 kB
    PageTables: 1804 kB
    VmallocTotal: 536870911 kB
    VmallocUsed: 279656 kB
    VmallocChunk: 536591019 kB
    HugePages_Total: 8900
    HugePages_Free: 8900
    Hugepagesize: 2048 kB
    
    # cat /proc/sys/vm/nr_hugepages 
    8900
  

• It can be seen that the system has been configured to keep  apart 8900 pages of memory for HugePages. At 2048 KB per page, that  translates to 18227200 KB, or about 17 GB of memory.

    # free -m
                   total used free shared buffers cached
    Mem:                32181 18267 13913 0      42      305
    -/+ buffers/cache:            17920 14261
    Swap: 24575 0 24575
  

• In the output of 'free -m', it can be seen that 18267 MB of  memory is given as used, and this includes the 17800 MB of memory  kept apart for HugePages. It seems that the Oracle DB has not been configured to use HugePages for the SGA, since they are remaining free. This has led to  the wastage of memory resources on the system, as the HugePages area of  memory is kept apart, but remains un-utilised.

• More information regarding HugePages can be got from the  article available at the link given below.

  • What are Huge Pages and what are the advantages of using them?

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.