Should the cache be dropped on a GFS2 file-system when a performance issue is occurring?

Environment

• Red Hat Enterprise Linux Server 5 (with the High Availability and Resilient Storage Add Ons)
• Red Hat Enterprise Linux Server 6 (with the High Availability and Resilient Storage Add Ons)
• A Global Filesystem 2(GFS2)

Issue

• Should the cache be dropped on a GFS2 file-system when a performance issue is occurring?

Resolution

Dropping of the pagecache, dentries and inodes should be used only for debugging.

# echo 3 > /proc/sys/vm/drop_caches

If GFS2 file-system is having performance issues then dropping of the pagecache, dentries and inodes should be used as last resort. Here are a couple of items that should be considered before dropping the pagecache, dentries and inodes:

• GFS2 uses caching and when the pagecache, dentries and inodes is dropped then there will be a performance penalty until the table is rebuilt.
• There could be a heavy performance hit when the cache is dropped on large tables that could be noticeable to end users.
• The duration of how long a cache drop takes is a function of computing power, the number(and type) of cached locks, and current activity on the file-system. These factors and their interactions are environment-specific and can only be established through empirical observation within a particular environment or workload.

When the pagecache, dentries and inodes drop the invalidation of locks will not linear. The majority of lock invalidation does not occur until glock_workqueue has walked the set of locks to be dropped. Thus, the rate of glock dropping is not linear with respect to time. In most cases there will be a huge drop of the objects in /proc/slabinfo when it is about to complete

Reference

• What are some examples of GFS & GFS2 workloads that should be avoided?
• What data should I gather when access to a GFS2 filesystem appears to be hung, unresponsive, or slow in RHEL?
• This content is not included.How do I debug a GFS/GFS2 performance problem?
• Is it possible to skip writing to cache for a GFS2 Filesystems?
• For more information on dropping the page cache then the review the Content from www.kernel.org is not included.kernel documentation on sysctl/vm.

Root Cause

The GFS2 file-system is having performance(or appears slow) issues when the file-system is being accessed. Poor performance on GFS2 in many cases is a function of an access pattern incompatible with the design of GFS2.

Diagnostic Steps

If dropping the page cache is needed then usually that indicates that there is I/O access pattern on GFS2 that should be reviewed.

Statistics on the number of dlm objects and the number of cached glocks can be found in /proc/slabinfo:

# sed -n '2p;/dlm_lkb/p;/glock/p' /proc/slabinfo

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.