How do I increase the kernel log ring buffer size?

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux 9
  • Red Hat Enterprise Linux 8
  • Red Hat Enterprise Linux 7
  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 5

Issue

  • How do I increase the kernel log ring buffer size?
  • Why are boot messages missing from the logs?
  • A lot of boot messages are missing from the logs.
  • The dmesg file does not contain all the log messages from boot.
  • I need to troubleshoot the boot process but some of the boot messages are missing.

Resolution

Increase the kernel log buffer, either permanently for every boot, or temporarily just for that boot.

Permanent Change

RHEL8 and later

Use grub2-editenv command to modify the kernel command line.
Refer: How do I permanently modify the kernel command line in RHEL 8 or 9?

RHEL7

Edit the /etc/default/grub file and add log_buf_len=8M to the end of the GRUB_CMDLINE_DEFAULT line.

For example, if the current default line reads:

GRUB_CMDLINE_LINUX="rd.lvm.lv=vg_root/lv_root"

Change the line so it reads:

GRUB_CMDLINE_LINUX="rd.lvm.lv=vg_root/lv_root log_buf_len=8M"

Rebuild the GRUB configuration used at boot time with the command:

grub2-mkconfig -o /boot/grub2/grub.cfg

Reboot the system

RHEL 6 and RHEL 5

Edit the /boot/grub/grub.conf file and add log_buf_len=8M to the end of the GRUB kernel line.

For example, if the current kernel line is:

  kernel /vmlinuz-2.6.32-696.el6.x86_64 ro root=/dev/mapper/vg_root-lv_root

Change the line so it reads:

  kernel /vmlinuz-2.6.32-696.el6.x86_64 ro root=/dev/mapper/vg_root-lv_root log_buf_len=8M

Reboot the system.

Temporary Change

When booting the system, stop the boot process on the GRUB kernel selection screen by pressing an arrow key up or down.

  • Press e to edit the boot entry
  • Use up and down arrows to select the kernel line with the vmlinuz file on it

RHEL 7

  • Press End to place the cursor on the end of that line
  • Add log_buf_len=8M to the end of the kernel line, as shown in the examples above
  • Press Ctrl+x to boot with the edited boot entry

RHEL 6 and RHEL 5

  • Press e again to edit the kernel line
  • Add log_buf_len=8M to the end of the kernel line, as shown in the examples above
  • Press Enter to temporarily save the changes
  • Press b to boot with the edited boot entry

Root Cause

To avoid losing boot messages, the kernel log buffer must be a large enough size to contain all the boot messages up until the point the system is able to write the log to permanent storage. This is especially true if additional logging is enabled; for example by removing quiet from the boot line or setting log_level=debug - or you have a large number of storage devices.

If the kernel log buffer is not large enough, the oldest messages during boot will be lost, overwritten by the latest messages.

For example, the following knowledgebase solutions show specific cases where a recommendation to increase the log_buf_len is present:

While there is no specific recommended minimum that would be applicable to all configurations, a reasonable practical amount for most configurations is 8M in size. Typically this is a small fraction of total physical memory on modern enterprise hardware. As such, the additional memory should not adversely affect system operations, while the additional ring buffer memory often enhances supportability if or when an issue related to boot time happens or within a vmcore at crash time as well under other circumstances.

By default, the log ring buffer size is set in the conf variable CONFIG_LOG_BUF_SHIFT.

To check the default size:

# grep CONFIG_LOG_BUF_SHIFT /boot/config*
CONFIG_LOG_BUF_SHIFT=20                                     # RHEL 7 and later
CONFIG_LOG_BUF_SHIFT=19                                     # RHEL 6 (2.6.32)
CONFIG_LOG_BUF_SHIFT=18                                     # RHEL 5 (2.6.18)

The CONFIG_LOG_BUF_SHIFT specifies the 2^N default size of the log ring buffer (1<<CONFIG_LOG_BUF_SHIFT):

  • 20: 2^20 = 1024KB (RHEL 7 and later default log ring buffer size)
  • 19: 2^19 = 512KB (RHEL 6 default log ring buffer size)
  • 18: 2^18 = 256KB (RHEL 5 default log ring buffer size)

The default size of log_buf_len for RHEL7 and later is ~1048576 bytes. Details explained in How much is the default log_buf_len size?

This default kernel configuration variable is set during compilation, and cannot be changed without recompiling the kernel (which is not supported). Overriding the default value requires changing the GRUB boot menu and rebooting. The buffer size may be increased by using the log_buf_len kernel boot parameter either for one boot by editing the boot menu at boot time, or permanently by changing the appropriate GRUB configuration file on disk.

The log_buf_len kernel parameter is described within the kernel-parameters.txt file:

log_buf_len=n Sets the size of the printk ring buffer, in bytes.
             Format: { n | nk | nM }
             n must be a power of two.  The default size is set in the kernel config file.
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.