Nested virtualization in OpenShift Virtualization

Environment

• Red Hat OpenShift Container Platform 4.15 or later
• Red Hat OpenShift Virtualization
• Running a guest virtual machine (VM) within an guest virtual machine

Issue

• Deploying guest VM within a guest VM

Resolution

IMPORTANT: Nested Virtualization is a Technology Preview only feature, so it has a reduced support scope. See the KCS article 58204 for more information about the nested virtualization support.

• Machine Configuration
  In recent OpenShift versions nested virtualization is enabled by default. You can verify it in the worker nodes by confirming that the nested parameter of the respective kvm module is 1 or Y:

  • For AMD hosts:

    # cat /sys/module/kvm_amd/parameters/nested
    1
    

  • For Intel hosts:

    # cat /sys/module/kvm_intel/parameters/nested
    1
    

  If nested virtualization is disabled, you can enable it by adding the option nested=1 to the kernel modules kvm_{amd,intel} in the relevant MachineConfigPool.

  • For example, the following MachineConfig manifest will create the file of /etc/modprobe.d/kvm.conf with the required contents and will trigger a reboot of all the worker nodes, one by one:

    apiVersion: machineconfiguration.openshift.io/v1
    kind: MachineConfig
    metadata:
      labels:
        machineconfiguration.openshift.io/role: worker
      name: 80-enable-nested-virt
    spec:
      config:
        ignition:
          version: 3.2.0
        storage:
          files:
          - contents:
              source: data:text/plain;charset=utf-8;base64,b3B0aW9ucyBrdm1faW50ZWwgbmVzdGVkPTEKb3B0aW9ucyBrdm1fYW1kIG5lc3RlZD0xCg==
            mode: 0644
            path: /etc/modprobe.d/kvm.conf
    

• Virtual Machine Configuration
  Ensure your CPU type is either host-model or host-passthrough or provide the Virtualization CPU flags as required to the VM.

  • For example, for any CPU :

    model: host-model  # Or: host-passthrough
    

  • For example, for AMD type CPU :

    features:
        - name: svm
          policy: require
    model: <CPU model>  # Optional
    

  • Or for Intel type CPU :

    features:
        - name: vmx
          policy: require
    model: <CPU model>  # Optional
    

Root Cause

Exposing the svm or vmx CPU flags in the L1 guest is required to run nested VMs.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.