I/O errors from unknown storage device after installing a new SAN network
Environment
- Red Hat Enterprise Linux 5
Issue
-
Just installed EMC software, EMCpower, NAVICli, and NAVIHostAgent. When the system boots the system logs show repeated kernel I/O errors.
Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2097096 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2097096 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2097112 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2097112 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2097128 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2097128 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2096896 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2096896 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2096904 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2096904 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2096920 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2096920 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2096936 Aug 2 03:40:24 host1 kernel: end_request: I/O error, dev sda, sector 2096936 -
The system is connected to the SAN but no disks have been assigned yet. The only disks right now are local :
[root@host1 tmp]# fdisk -l Disk /dev/cciss/c0d0: 293.5 GB, 293564211200 bytes 255 heads, 63 sectors/track, 35690 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/cciss/c0d0p1 * 1 33 265041 83 Linux /dev/cciss/c0d0p2 34 35690 286414852+ 8e Linux LVM
Resolution
- Check if any Luns are exported and confirm with the storage vendor.
Root Cause
- There are some storage devices that have a special "test" Lun that is used for testing. This especially true for most Sun Storagetek Arrays.
Diagnostic Steps
-
The disk are exported from the SAN storage :
Aug 2 04:12:45 host1 kernel: Emulex LightPulse Fibre Channel SCSI driver 8.2.0.63.1p Aug 2 04:12:45 host1 kernel: Copyright(c) 2004-2009 Emulex. All rights reserved. Aug 2 04:12:45 host1 kernel: GSI 22 sharing vector 0xAA and IRQ 22 Aug 2 04:12:45 host1 kernel: ACPI: PCI Interrupt 0000:0b:00.0[A] -> GSI 31 (level, low) -> IRQ 170 Aug 2 04:12:45 host1 kernel: scsi2 : on PCI bus 0b device 00 irq 170 Aug 2 04:12:45 host1 kernel: GSI 23 sharing vector 0xB2 and IRQ 23 Aug 2 04:12:45 host1 kernel: ACPI: PCI Interrupt 0000:08:00.0[A] -> GSI 32 (level, low) -> IRQ 178 Aug 2 04:12:45 host1 kernel: scsi3 : on PCI bus 08 device 00 irq 178 Aug 2 04:12:45 host1 kernel: lpfc 0000:0b:00.0: 0:1303 Link Up Event x1 received Data: x1 xf7 x10 x9 x0 x0 0 Aug 2 04:12:45 host1 kernel: Vendor: DGC Model: LUNZ Rev: 0429 Aug 2 04:12:45 host1 kernel: Type: Direct-Access ANSI SCSI revision: 04 Aug 2 04:12:45 host1 kernel: sda : READ CAPACITY failed. Aug 2 04:12:45 host1 kernel: sda : status=1, message=00, host=0, driver=08 Aug 2 04:12:45 host1 kernel: sd: Current: sense key: Illegal Request Aug 2 04:12:45 host1 kernel: Add. Sense: Logical unit not supported Aug 2 04:12:45 host1 kernel: Aug 2 04:12:45 host1 kernel: sda: test WP failed, assume Write Enabled Aug 2 04:12:45 host1 kernel: sda: asking for cache data failed Aug 2 04:12:45 host1 kernel: sda: assuming drive cache: write through Aug 2 04:12:45 host1 kernel: sda : READ CAPACITY failed. Aug 2 04:12:45 host1 kernel: sda : status=1, message=00, host=0, driver=08 Aug 2 04:12:45 host1 kernel: sd: Current: sense key: Illegal Request Aug 2 04:12:45 host1 kernel: Add. Sense: Logical unit not supported Aug 2 04:12:45 host1 kernel: Aug 2 04:12:45 host1 kernel: sda: test WP failed, assume Write Enabled Aug 2 04:12:45 host1 kernel: sda: asking for cache data failed Aug 2 04:12:45 host1 kernel: sda: assuming drive cache: write through -
After initializing the Emulex HBA, A LUN is detected and attempt is made to add it to the system. But this seem to result in an illegal request error from the target i.e. the storage.
SBR
Product(s)
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.