What is the theoretical maximum number of sd* devices supported?

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux 4
  • Red Hat Enterprise Linux 5
  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 7
  • Emulex HBA
  • QLogic HBA

Issue

  • Total SCSI disk devices
  • Targets per HBA
  • LUNs per target
  • Maximum multipath paths

Resolution

There are several layers of limitations that can affect the maximum number of LUNs supported. LUNs can include disks, tapes, changers, and other types of devices.

  • LUNs can be counted two different ways:

    • by unique SCSI address, the following is 4 LUNs. This is the more typical method and the one used within this document.
      2:0:1:21  sdbv 68:144
  2:0:2:21  sdly 69:256
  3:0:1:21  sdbs 68:96 
  3:0:2:21  sdlx 68:496
    • by unique LUN WWID/ID, the following is 1 LUN (and 4 paths to that LUN, and 4 named sd* disks).
               +--- WWID
           v
  mpathax (3600140569acc52261324121a82af9350) dm-30 LIO-ORG ,testvol1        
    size=10G features='0' hwhandler='0' wp=rw
    |-+- policy='service-time 0' prio=1 status=active
    | `- 2:0:1:21  sdbv 68:144  active ready running
    | `- 2:0:2:21  sdly 69:256  active ready running
    | `- 3:0:1:21  sdbs 68:96   active ready running
      `- 3:0:2:21  sdlx 68:496  active ready running
               ^
               +--- LUN ID

  • The SCSI block (disk) device names can run from /dev/sda to /dev/sdzzz. This is a maximum of 18,278 devices. As a practical matter, this is the nominal upper limit of how many scsi disks/LUNs that can be supported.. A system configuration will usually hit the disk naming limit before hitting other kernel maximum (tuned) limits.
    This is not the limit of scsi LUNs, just disk LUNs, as other types of scsi devices such as tapes, changers, etc. have their own naming conventions and name space limits.
    1） sda ~ sdz : 26
    2） sdaa ~ sdzz : 26x26=676
    3） sdaaa ~ sdzzz: 26x26x26=17576
    4） total=26+26x26+26x26x26=18278

    • This limit cannot be increased or decreased.

  • The SCSI mid layer supports up to a maximum of 0xffffffff (4294967295) luns, but is set to a lower number by default.

  • The maximum supported LUN count can sometimes be capped due to limits within the HBA driver. Depending on the driver, the configured limit may need to be increased for larger configurations.

    • Emulex controllers support a maximum of 65,535 LUNs. The default is set to 255.

    • QLogic controllers support a maximum of 65,535 LUNs. The default is set to 65,535

      • see current limit: cat /sys/module/qla2xxx/parameters/ql2xmaxlun

    • The above are two common FC drivers. Other drivers may have similar limits, but many drivers have a fixed limit set within the driver than cannot be changed.

  • device-mapper-multipath and the kernel support up to 1024 path groups with up to 1024 paths per group for each multipath map.

Diagnostic Steps

  • The default setting for lpfc.lpfc_max_luns (Emulex HBAs) is 255. This can be checked with the following command.

      # cat /sys/module/lpfc/parameters/lpfc_max_luns 
  255

  • The default setting for qla2xxx.ql2xmaxlun is 65535. This can be checked with the following command (RHEL5 and above only).

      # cat /sys/module/qla2xxx/parameters/ql2xmaxlun 
  65535

  • The default setting for scsi_mod.max_luns (SCSI mid layer) is 512. This can be checked with the following command.

      # cat /sys/module/scsi_mod/parameters/max_luns
  512
SBR
Product(s)
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.