Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux 9
  • Red Hat Enterprise Linux 8
  • Red Hat Enterprise Linux 7
  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 5
  • Red Hat Enterprise Linux 4
  • Oracle 9g
  • Oracle 10g
  • Oracle 11g
  • Oracle 12c, 18c, 19c

Issue

  • How can I tune my Red Hat Enterprise Linux system for Oracle 10g/11g?
  • High memory consumption on Oracle database system
  • High CPU consumption on Oracle hosts
  • How should I configure swap memory for an Oracle database?
  • Slower than expected IO performance running Oracle on RHEL
  • What is the maximum number of huge pages I should be using per system?
  • Should I disable numa_balancing ?

Resolution

The information contained herein is just an example of how the configuration may look like and is provided just for the convenience only. See Reference section for additional examples and discussion reguarding tuning.

  In the absence of any future documentation from Red Hat specifically for RHEL8 and later, the recommendations for RHEL7 are applicable for newer kernel versions.

  • IMPORTANT: This article is not an authoritative source of information on how to configure Red Hat Enterprise Linux for use with Oracle products, as these products are not shipped by Red Hat. The authoritative source of such information is the Oracle Installation Guide or Oracle OS Tuning Guide for your version of Oracle product, such a documentation could be found on Oracle MetalinkThis content is not included.[1] site, see the following Notes available on the Oracle Metalink site.

    • RHEL4 32 bit (x86):

      • Note 303859.1 - Requirements for Installing Oracle 9iR2 on RHEL 4
      • Note 392940.1 - Requirements for Installing Oracle 10.1.0.x RDBMS on RHEL 4 x86 platform
      • Note 343431.1 - Requirements for Installing Oracle 10gR2 RDBMS on RHEL 4 x86 platform
      • Note 430653.1 - Requirements for Installing Oracle 11gR1 32-bit on RHEL 4
      • Note 880211.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 4 x86
    • RHEL4 64 bit (x86-64):

      • Note 353529.1 - Requirements for Installing Oracle 9iR2 64-bit on RHEL 4 x86-64 (AMD64/EM64T)
      • Note 390900.1 - Requirements for Installing Oracle 10g (10.1.0.x) RDBMS on RHEL 4 on AMD64/EM64T (Linux x86-64)
      • Note 339510.1 - Requirements for Installing Oracle 10gR2 RDBMS on RHEL 4 on AMD64/EM64T
      • Note 437123.1 - Requirements for Installing Oracle 11gR1 RDBMS on RHEL 4 on AMD64/EM64T
      • Note 880942.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 4 on AMD64/EM64T
    • RHEL5 32 bit (x86):

      • Note 419646.1 - Requirements For Installing Oracle 10gR2 On RHEL 5 (x86)
      • Note 438765.1 - Requirements for Installing Oracle 11gR1 32bit RDBMS on RHEL 5
      • Note 880936.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 5 on 32-bit x86
    • RHEL5 64 bit (x86-64):

      • Note 421308.1 - Requirements For Installing Oracle10gR2 On RHEL 5 (x86_64)
      • Note 438766.1 - Requirements for Installing Oracle 11gR1 RDBMS on RHEL 5 on AMD64/EM64T
      • Note 880989.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 5 on AMD64/EM64T
      • Note 1529433.1 - Requirements for Installing Oracle Database 12.1 on RHEL5 or OL5 64-bit (x86-64)
    • RHEL6 64 bit (x86-64):

      • Note 1441282.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL6 or OL6 64-bit (x86-64)
      • Note 1529864.1 - Requirements for Installing Oracle Database 12.1 on RHEL6 or OL6 64-bit (x86-64)
      • Note 2426831.1 - Requirements for Installing Oracle Database 18c on OL6 or RHEL6 64-bit (x86-64)
    • RHEL7 64 bit (x86-64):

      • Note 1962100.1 - Requirements for Installing Oracle 11.2.0.4 RDBMS on OL7 or RHEL7 64-bit (x86-64)
      • Note 1961997.1 - Requirements for Installing Oracle Database 12.1on RHEL7 or OL7 64-bit (x86-64)
      • Note 2426806.1 - Requirements for Installing Oracle Database 18c on OL7 or RHEL7 64-bit (x86-64)
      • Note 2551169.1 - Requirements for Installing Oracle Database 19c on OL7 or RHEL7 64-bit (x86-64)
    • Content from blogs.oracle.com is not included."Master Note of Linux OS Requirements for Database Server"

There are 3 ways to tune Oracle database server:

Refer Red Hat kcs: Installing tuned profile available for Oracle RDBMS

2. Automatic configuration

Red Hat provides a This content is not included.RHEL Tuner for Oracle tool to help tune your Red Hat Enterprise Linux for Oracle. The tool incorporates the information included in this document but makes it easier to generate valid and support-recommended configurations. You can leave feedback on the tool at This content is not included.RHEL Tuner for Oracle.

3. Manual configuration

  • Note: All the numbers mentioned in this article are not generic and have been shown to have a positive effect on database workloads resident on Red Hat Enterprise Linux. That being said, the specific values depend on user environments and may require further adjustment.

Memory settings in /etc/sysctl.conf:

  • Swapping for Oracle is not ideal and should be avoided as much as possible. The following tunable will tune the kernel to swap less aggressively.

      vm.swappiness=10
    
  • Maximum percentage of active memory that can have dirty pages. For example if a system has 1000 pages of memory and dirty_background_ratio is set to 3%, writeback will begin when 30 pages have been dirtied.

      vm.dirty_background_ratio=3
    
  • Maximum percentage of total memory that can have dirty pages. If it is set to 40% on a 1000 page system, a process dirtying pages will be made to wait once the 400th page is dirtied.This mechanism will, thus, slow the dirtying of pages while the system catches up.

      vm.dirty_ratio=40
    
  • How long data can be in page cache before being expired:

      vm.dirty_expire_centisecs=500
    
  • How often pdflush is activated to clean dirty pages (in hundreds of a second):

      vm.dirty_writeback_centisecs=100
    

HugePages

  1. Obtain the Hugepagesize from /proc/meminfo

     $ grep Huge /proc/meminfo
     HugePages_Total:       0
     HugePages_Free:        0
     HugePages_Rsvd:        0
     Hugepagesize:       2048 kB
    
  2. Calculate the recommended number of HugePages using the formula below, using Hugepagesize and the System Global Area (SGA) values from Oracle. See Oracle documentation for more about SGA and where to find those values. Note that the Program Global Area (PGA) does not use huge pages.

     SGA / huge page size 
    
    • The following is an example showing that 10240 is amount of huge pages required to fit the 20GB SGA. Please note, that values were are all converted to kilobytes for calculation:

        Hugepagesize = 2048 KB
        20 GB SGA = 20971520 KB
      
        (20971520 / 2048) = 10240 
      
  3. Add that number to the /etc/sysctl.conf setting:

     vm.nr_hugepages=10240
     vm.hugetlb_shm_group=<insert oracle group ID here>
    
  4. Once the above setting has been tested and shown to provide the performance required, it is recommended to move HugePage allocation to the kernel line in grub configuration as a boot option. This allows for earlier and cleaner allocation of HugePages:

     kernel /vmlinuz-<kernel-version> ro root=/dev/vg01/lv01 hugepages=10240 <other boot options>
    

    How to permanently modify the kernel command line in RHEL5 and RHEL6?
    How do I permanently modify the kernel command line?

    • If Hugepages are not set in kernel command line, they can be dynamically allocated by setting the value in nr_hugepages.
    • When the system is under memory pressure, the huge pages cannot be swapped.
  5. Change the Oracle database configuration to use the Huge Pages. Contact Oracle support if assistance is needed

Limits setting in /etc/security/limits.conf:

  • Note: In order for Oracle database to use Huge Pages in Red Hat Enterprise Linux, you may also need to increase the ulimit parameter "memlock" for the oracle user in /etc/security/limits.conf. The memlock setting is specified in KB and must match the memory size of the number of Huge Pages that Oracle should be able to allocate. So, if the Oracle database should be able to use 512 Huge Pages, then memlock must be set to at least (512 * Hugepagesize), which on a default system would be 1048576 KB (512*2048).

      oracle     soft     memlock     1048576  
      oracle     hard     memlock     1048576
    
  • The memlock behavior for SHM_HUGETLB is changed from RHEL 9.2 onwards, please check following article for additional details.
    What will happen if huge pages are configured but sysctl parameter vm.hugetlb_shm_group remains set to default value?.

Shared Memory

  • According to the Content from www.kernel.org is not included.documentation:

    • SHMALL is the maximum total amount of shared memory pages that can be used system-wide. Hence, SHMALL should always be at least ceil(SHMMAX/PAGE_SIZE).
    • SHMMAX is the maximum size in bytes of a single shared memory segment that can be created.
  • Obtain the total memory in the system in bytes:

      mem=$(free -b | awk '/Mem/ {print $2}')
    
  • Get page size in bytes:

      page=$(getconf PAGE_SIZE)
    
  • Calculate 75% of the total memory in the system in pages for SHMALL:

      all=$(expr $mem \* 75 / 100 / $page + 1)
    
  • Multiply the SHMALL value by the page size to get SHMMAX:

      max=$(expr $all \* $page)
    
  • Set the SHMMAX and the SHMALL values in the /etc/sysctl.conf file:

      echo "kernel.shmmax = $max" >> /etc/sysctl.conf
      echo "kernel.shmall = $all" >> /etc/sysctl.conf
    
  • Set the maximum number of shared memory segments with SHMMNI in the /etc/sysctl.conf file:

      echo "kernel.shmmni = 4096" >> /etc/sysctl.conf
    

Semaphores

  • Recommended minimums for semaphore operations:

      kernel.sem="250 32000 100 128"
    
  • The first value, SEMMSL, is the maximum number of semaphores per semaphore set

  • The second value, SEMMNS, defines the total number of semaphores for the system

  • The third value, SEMOPM, defines the maximum number of semaphore operations per semaphore call

  • The last value, SEMMNI, defines the number of entire semaphore sets for the system

      # sysctl -w "kernel.sem = 250 32000 100 128"
      # echo "kernel.sem = 250 32000 100 128" >> /etc/sysctl.conf
    

Open file descriptors for oracle user

    # vi /etc/security/limits.conf

    #<domain>      <type>  <item>         <value>
    oracle          hard    nofile          10000

Disabling transparent hugepages (THP) recommended just for Red Hat Enterprise Linux 6 and 7 versions

  • Disable tuned and ktune services if installed

    • Example #1:

       # service tuned stop
       # chkconfig tuned off
       # service ktune stop
       # chkconfig ktune off
      
    • Example #2:

       # tuned-adm off
      

Note: The tuned-adm command will revert all your settings to what they were before tuned started and disable the tuning services from running at boot. As an alternative, a customized tuned profile can be created, taking over the old settings and only disabling THP. Refer to kbase Disabling transparent hugepages (THP) on Red Hat Enterprise Linux 6 is not taking effect. for details.

  • Append transparent_hugepage=never to kernel command line in /boot/grub/grub.conf file. For example:

      kernel /boot/vmlinuz-2.6.32-358.el6.x86_64 ro root=UUID=a216d1e5-884f-4e5c-859a-6e2e2530d486 rhgb quiet transparent_hugepage=never
    
  • Rebuild the grub:

# grub2-mkconfig -o /boot/grub2/grub.cfg
  • For RHEL 8 and later
# grubby --update-kernel=/boot/vmlinuz-$(uname -r) --args="<NEW_PARAMETER>"

Example: # grubby --update-kernel=/boot/vmlinuz-$(uname -r) --args="transparent_hugepage=never"
  • Reboot the server for changes to take effect.

      # reboot
    

I/O scheduler

  • The default CFQ I/O scheduler is appropriate for most workloads, but does not offer optimal performance for database environments. Instead, one of the following IO schedulers should be used:

NUMA balancing

  • Auto NUMA balancing
  • By default on RHEL7 numa_balancing is enabled. This can see an impact for Oracle workload so the best practice is to prevent the NUMA balancing.
    • configure kernel.numa_balancing = 0 in /etc/sysctl.conf
    • run: sysctl -p

Reference

[1] Metalink is now referenced as "My Oracle Support". This is Oracle's official electronic online support serves portal. Users will need a valid Oracle support contract in order to login and read the referenced Notes herein.
the

SBR
Components
Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.