All about resource limits: ulimit, pam_limits.so, /etc/limits.conf, and /etc/limits.d/
Introduction
System administrators try to balance the system load with the hardware resources they have available.
The pam_limits and its configuration files are a rudimentary method of setting limits on a per process basis. Administrators mistake limits.conf ability to restrict processes resources to a per-user or per-session basis.
Why Do We Need Limits?
Pam_limits module acts as a resource controller for individual processes. Resource limits can prevent over-use of resources in certain situations such as:
- An application that goes 'haywire' and continually spawns many processes , termed
fork bombing - A process opening too many files at once leading to crippled system performance
- A process triggering denial-of-service attack on a system by consuming a finite resource.
System wide settings exist to set upper limits for every process. An administrator can use pam_limits to provide a granular approach to control resource usage.
How it works
A PAM module called pam_limits sets resource limits when a user authenticates using the PAM stack. It is important to note that limits are set per process. Modification to PAMs limits are not retroactively applied to existing sessions.
The ulimit command
The ulimit command allows users to view or reduce their limits for the current shell. Only the root user can increase restrictions above the specified hard limit The limits set also apply to any child process of the shell. The shell man page provides more information about usage of the ulimit command, an excerpt from the bash man page is below:
ulimit [-HSTabcdefilmnpqrstuvx [limit]]
Provides control over the resources available to the shell and to processes started by it, on systems that allow such control.
Options are interpreted as follows:
-a All current limits are reported
-b The maximum socket buffer size
-c The maximum size of core files created
-d The maximum size of a process’s data segment
-e The maximum scheduling priority ("nice")
-f The maximum size of files written by the shell and its children
-i The maximum number of pending signals
-l The maximum size that may be locked into memory
-m The maximum resident set size (many systems do not honor this limit)
-n The maximum number of open file descriptors (most systems do not allow this value to be set)
-p The pipe size in 512-byte blocks (this may not be set)
-q The maximum number of bytes in POSIX message queues
-r The maximum real-time scheduling priority
-s The maximum stack size
-t The maximum amount of cpu time in seconds
-u The maximum number of processes available to a single user
-v The maximum amount of virtual memory available to the shell
-x The maximum number of file locks
-T The maximum number of thread
Persistent limits
Persistent limits are set using either of the following methods:
Limits Configuration Files
The pam_limits module applies limits from the system wide /etc/security/limits.conf file.
Pam_limits also sources configuration from the .conf files in the /etc/security/limits.d/ directory. The configuration in these files take precedence over the system wide configuration in /etc/security/limits.conf
For example:
$ cat /etc/security/limits.d/90-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nproc 1024
mysql soft nproc 1064960
mysql hard nproc 1064960
root soft nproc unlimited
Shell Startup Files
Limits can be set as shell commands in the shell initialization files. The most common files are /etc/profile and ~/.bash_profile.
Users can sidestep this limitation if they have permission to change or avoid execution of these files.
A sample limitation:
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
System wide limits take affect when limits are not specified in the configuration files loaded by pam_limits.so or any shell startup files.
Hard and Soft Limits
The soft and hard configuration keyword have explicit meaning. Soft limits are the currently enforced limit while hard limits are the upper bounds of a resource that a process can use.
Exercise caution when configuring a soft limit. Increasing a configuration value above the soft limit requires manual interaction. Manual change may cause problems for user accounts that are only logged in and currently in use by applications.
The ulimit -Ha and ulimit -Sa commands output the current hard and soft limits respectively.
It can be beneficial to set the hard and soft limit to the same value using the - character in limits.conf
mysql - nproc 1064960
Process limits
When a process is created it inherits the value set by the parent process. The proc filesystem provides the ability to see what the current process limits are for any process.
[root@rhel64 ~]# cat /proc/`pidof crond`/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 10485760 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 7819 7819 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 7819 7819 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Depending on the initscript, some services started on boot will not utilize limits as you might expect:
Refer: How to set limits for services started at boot
Why are limits creating problems for my application
Messages such as the errors below for some resource-intensive applications such as Oracle due to hitting a limit constraint.
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: <some-string>
Red Hat recommends a high limit for common resources such as nproc and nofile for your application needs.