Need help understanding MALLOC_ARENA_MAX

Environment

Red Hat Enterprise Linux (RHEL) 6

Issue

More information about how an arena functions

Resolution

An arena is a scratchpad that glibc maintains to return smaller blocks to the requestor. As you may have guessed, the data area (or ‘heap’) created by extending the process break (using the brk syscall) is also an arena – it is referred to as the main arena.

In addition to the main arena, glibc malloc allocates additional arenas. The reason for creation of arenas always seems to have been to improve performance of multithreaded processes. Multiple arenas implementation did the following to reduce this contention:

• Firstly, the malloc call always tries to always stick to the arena it accessed the last time
• If the earlier arena is not available immediately (as tested by a pthread_mutex_trylock), try the next arena
• If we don’t have uncontended access on any of the current arenas, then create a new arena and use it.

There is a limit to the number of arenas that are created in this manner and that limit is determined based on the number of cores the system has. 32-bit systems get twice the number of cores and 64-bit systems get 8 times the number of cores. This can also be controlled using the MALLOC_ARENA_MAX environment variable.

A sane number of threads is usually not more than twice the number of cores. Anything more and you’ve have a different set of performance problems to deal with.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.