Disabling transparent hugepages (THP) on Red Hat Enterprise Linux 6 is not taking effect.

Environment

• Red Hat Enterprise Linux (RHEL) 6
• transparent hugepages (THP)
• tuned
• ktune

Issue

• Unable to disable transparent hugepages (THP) even after appending "transparent_hugepage=never" to kernel command line in /boot/grub/grub.conf file.
  Eg:

# grep -i never /boot/grub/grub.conf 
	kernel /boot/vmlinuz-2.6.32-358.el6.x86_64 ro root=UUID=a216d1e5-884f-4e5c-859a-6e2e2530d486 rhgb quiet transparent_hugepage=never

# cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
[always] never

# grep -i AnonHugePages /proc/meminfo 
AnonHugePages:    206848 kB

Resolution

Create a customized tuned profile with disabled THP

With this resolution we will create a customized version of the currently running profile. The customized version will disable THP.

Find out which profile is active, create a copy. In the following example we currently use the throughput-performance profile:

$ tuned-adm  active
Current active profile: throughput-performance
Service tuned: enabled, running
Service ktune: enabled, running
$ cd /etc/tune-profiles/
$ cp -r throughput-performance throughput-performance-no-thp

We will now disable THP in the new profile and activate the new profile:

$ sed -ie 's,set_transparent_hugepages always,set_transparent_hugepages never,' \
      /etc/tune-profiles/throughput-performance-no-thp/ktune.sh
$ grep set_transparent_hugepages /etc/tune-profiles/throughput-performance-no-thp/ktune.sh
        set_transparent_hugepages never
$ tuned-adm profile throughput-performance-no-thp
$ cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
always [never]

We will also have to add the transparent_hugepage=never parameter to the kernel line /etc/grub.conf to ensure that when the system is booted applications already do not request THP, for example:

kernel /boot/vmlinuz-2.6.32-358.el6.x86_64 ro root=UUID=a216d1e5-884f-4e5c-859a-6e2e2530d486 rhgb quiet transparent_hugepage=never

The system should then be rebooted.

Alternative Resolution: Disable tuned and ktune services.

This resolution will disable the tuned/ktune services.

Eg:

# service tuned stop
# chkconfig tuned off
# service ktune stop
# chkconfig ktune off

OR

# tuned-adm off

Note: The tuned-adm command will revert all your settings to what they were before tuned started and disable the tuning services from running at boot.

• Append "transparent_hugepage=never" to kernel command line in /boot/grub/grub.conf file.
  Eg:

kernel /boot/vmlinuz-2.6.32-358.el6.x86_64 ro root=UUID=a216d1e5-884f-4e5c-859a-6e2e2530d486 rhgb quiet transparent_hugepage=never

• Reboot the server for changes to take effect.
  Eg:

# reboot

Root Cause

• The ktune service enables transparent hugepages (THP) by default for all profiles.

# cat /etc/tune-profiles/enterprise-storage/ktune.sh 
#!/bin/sh

. /etc/tune-profiles/functions

start() {
	set_cpu_governor performance
	set_transparent_hugepages always  <<<----
	disable_disk_barriers
	multiply_disk_readahead 4

	return 0
}

stop() {
	restore_cpu_governor
	restore_transparent_hugepages
	enable_disk_barriers
	restore_disk_readahead

	return 0
}

process $@

• Any time a tuned profile is enabled, whether at bootup via an init.d service, or manually at the command line, transparent hugepages (THP) gets re-enabled.

Diagnostic Steps

• Verify ktune and tuned services:

# chkconfig --list |egrep -i "ktune|tuned"
ktune          	0:off	1:off	2:off	3:on	4:on	5:on	6:off
tuned          	0:off	1:off	2:on	3:on	4:on	5:on	6:off

• When transparent_hugepage=never is appended to /boot/grub/grub.conf and the system is rebooted, the /sys/kernel/mm/transparent_hugepage/defrag option may still be enabled (ie: set to always). This can safely be ignored as THP is disabled, and THP defrag will not come into effect:

cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
always [never]

cat /sys/kernel/mm/transparent_hugepage/defrag
[always] never

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.