Reducing the MSI-X vector usage for a physical function to free vectors for associated virtual functions

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux 9
  • Red Hat Enterprise Linux 10

Issue

Not enough vectors available for virtual functions (VF)

Resolution

When managing network device resources, you might need to limit the number of Message Signaled Interrupts eXtended (MSI-X) vectors allocated to a physical function (PF) to ensure that enough vectors are available for associated virtual functions (VFs).

Note that only reducing the number of queues does not solve this issue, because the driver does not automatically adjust the MSI-X vector count.

  1. Set the number of queues for the network interface, for example:

    # ethtool -L <interface_name> combined 32

  2. Set the maximum number of MSI-X vectors:

    # echo 35 > /sys/bus/pci/devices/0000:xx:yy.z/msix_vec_per_pf_max

    The value for msix_vec_per_pf_max must be the number of queues you set in the previous step, plus three.

  3. Reload the device configuration to prompt the PF to correctly allocate the reduced number of MSI-X vectors:

    # devlink dev reload <devlink_device_path>

The PF now uses the specified number of MSI-X vectors, and the remaining vectors are freed up for use by VFs.

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.