How to change JVM memory options using Openshift images with Java?

Environment

• Red hat OpenShift Container Platform (OCP)
  • 4.x
  • 3.x
• Red Hat Middleware for Openshift Containers
  • JBoss Enterprise Application Platform (EAP)
    • 6.4.x
    • 7.x
  • JBoss Web Server (JWS)
    • 3.x
    • 5.x
  • JBoss Data Virtualization
    • 6.x
  • JBoss BRMS Decision Server
    • 6.x
  • JBoss BPM Process Server
    • 6.x
  • Red Hat SSO
    • 7.x
  • Red Hat Process Automation Manager
    • 7.x
  • Red Hat Decision Manager
    • 7.x

Issue

• How to customize Xms/Xmx memory params for JBoss JVM?
• Adding a JAVA_OPTS env variable in deploy config with memory sizings causes other values in JAVA_OPTS to be incorrect.

Resolution

As main recommendation: use container awareness

Recommend approach - set at deployment/deploymentconfig via resource constraints

Red hat products use OpenJDK images as basis, which has java container aware enabled by default and sets the maximum java heap size as a percentage of the container size (and the initial java heap size as a ratio of the maximum java heap size). So the the recommended approach is to use a higher level (runtime) resource constraints: deployment, or deploymentconfig, or statefulset. Another option is to set a limit-range.

Example Usage of resource constraint

Add a compute resource constraint for memory limits and requests to a DeploymentConfig/Deployment, which will then impose the limit on the pod:

$ oc edit dc <DC_NAME> <--- dc here means deploymentconfig

      spec:
        containers:
        - env:
          resources:
            limits:
              memory: 2G
            requests:
              memory: 1G

Refer to the This content is not included.Openshift Cluster Administration guide for examples of creating, viewing and deleting limit ranges.

Operator - recommended approach

As explained thorough in the solution Setting Env variables in JWS and JBoss EAP Operator in Openshift 4, when dealing with Operators, usually the user set this in the respective custom resources (CR) and not in th edeploymentconfig/deployment deployment.
In other words, for Operators the Custom Resource spec should be updated: Keycloack CR for SSO Operator, Infinispan Cr for Data Grid Operator, WildFlyServer CR for the EAP Operator.

Not recommended approach - setting at ram/java heap at pod level or overwriting the default ratio

Although it is possible to use the JAVA_OPTS or JAVA_OPTS_APPEND environment variable override/add JVM options to a pod or deployment configuration and provide customized JVM options, this approach is not recommended as it may result in the java heap (or native) consuming more than allocated on the container and a cgroups OOMEKill happening.

Note that directly setting the maximum/initial java heap size with -Xmx/-Xms respectively will override any JAVA_MAX_MEM_RATIO, JAVA_INITIAL_MEM_RATIO, and JAVA_MAX_MEM_RATIO settings, as the JVM will ignore the resulting -XX:InitialRAMPercentage/-XX:MaxRAMPercentage options.

The initial java heap defaults to 25% of the maximum java heap to support returning memory back to the container, if possible. OpenJDK garbage collectors that support returning memory back to the OS/container (e.g. shenandoah, parallel, serial, zgc, g1 on JDK17+) enable the functionality only if the initial java heap size is less than the maximum java heap size. This is the "low footprint" use case to support metered container environments. Setting the initial java heap to the maximum prevents the JVM from giving back memory, even if it is no longer needed (this is the "throughput" use case).

The solution How to modify JVM options (JAVA_OPTS) for JBoss EAP in Openshift discusses using the JAVA_OPTS_APPEND environment variable (and JWS as well).

Default values

By default, given the java container awareness, the xPaas JBoss EAP images will assign 50% of available memory in a container for java heap when compute resource constraints are in place (50% for java heap/50% for off-heap on ubi8 images, 80%/20 for ubi9). See details on Usage of Java flags InitialRAMPercentage and MaxRAMPercentage. See table below.

JAVA_MAX_MEM_RATIO and JAVA_INITIAL_MEM_RATIO can be used to set the java heap size. One can increase the maximum java heap size using the variables JAVA_MAX_MEM_RATIO or to change the java initial heap size, can change the value of JAVA_INITIAL_MEM_RATIO (which defaults to 25) as a DeploymentConfig environment variable. Be aware that java heap memory size is just one part of the total memory in a container, there are more spaces, like metaspace and the GC allocation and setting an excessive limit of JAVA_MAX_MEM_RATIO will trigger an OOME killer within the pod, eventually.

The following environment variables are deprecated:

*JAVA_OPTIONS is now an alias for JAVA_OPTS, which overrides (replaces) the image default JVM options. If it is desired to append to the image default JVM options, not replace, use JAVA_OPTS_APPEND.

Root Cause

Resource constraints

User can set resource constraints to change the desired memory values. Compute resource constraints can be used to specify how much CPU and memory each container has available to use. Compute resource constraints can be applied to pods, deployment configurations as well as templates. Alternatively, compute resource constraints can be applied via limit ranges, which enumerate compute resource constraints in a project at the pod, container, image and image stream level, and specify the amount of resources that a pod, container, image or image stream can consume.

In this matter, before Java was container aware those variables, .e.g JAVA_OPTS_APPEND, offered the safeguards to prevent an EAP container from requesting all the memory on a container host. Therefore, it is not recommend applying JVM heap settings using JAVA_OPTS_APPEND.

Java is container aware

Java is container aware so the max java heap size is deduced as half of the container size, which is set on the deployment config (for deployment config deployments) or set on the custom resources.
The max and min are automatically deduced, max will be half (50% percent) of the max container size, and 25% of the max will be the initial java heap size. So the initial java heap size is 25% of the max, because that's the ratio:

Thus, setting the flag JAVA_INITIAL_MEM_RATIO=50, in the deploymentconfig sets the initial java heap size to 50% of the maximum java heap size, instead of default 25% (of the maximum java heap size).
Although the java heap percentage can be altered by setting the JAVA_MAX_MEM_RATIO/CONTAINER_HEAP_PERCENT, which is an environment variable in a pod definition or deployment configuration that represents a percentage value of the pod that should be allocated to the java heap, the CONTAINER_HEAP_PERCENT environment variable is deprecated in favor of JAVA_MAX_MEM_RATIO/JAVA_INITIAL_MEM_RATIO. Just as example, CONTAINER_HEAP_PERCENT=0.50 represents 50% of the java heap.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.