CSE-216BE1C-R920WB hotswap crash

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

gtech1

Member
May 27, 2019
78
6
8
I purchased a brand new CSE-216BE1C-R920WB from Wiredzone and I populated it with a dual Xeon X10 motherboard, V4 cpus and LSI 9311-8i HBA.

All the components are brand new and the server has been working flawlessly for the past 60 days but last night I wanted to add 2 new drives to it while it was running and the OS completely freaked out.

I use FreeBSD 12.2-p6 in 6 other similar servers and adding drives via hotswap was never an issue.

When I added the 2 drives last night, all lights on the 24 bays turned red, including on the drives that were already active. Then all the lights turned off one by one, and the two drives I just inserted remained turned on with a red light.

Inside the OS, FreeBSD completely freaked out and nothing short of a hard reboot brought everything back up to normal. There was no data corruption and the new drives were seen properly by the system after reboot.

I'm suspecting a hardware issue because of the red lights coming on but what can it be ? The LSI HBA also has the latest firmware:

mpr0: <Avago Technologies (LSI) SAS3008> port 0x5000-0x50ff mem 0xc7240000-0xc724ffff,0xc7200000-0xc723ffff irq 32 at device 0.0 numa-domain 0 on pci4
mpr0: Firmware: 16.00.01.00, Driver: 23.00.00.00-fbsd


This is what FreeBSD reported at the time of the crash:

May 21 20:04:20 dfs12 kernel: mpr0: SMP command timed out during discovery for expander with SAS Address 500304801f7bfdff and handle 0x9.
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 10
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 11
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 12
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 13
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 14
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 15
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 16
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 17
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 18
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 19
May 21 20:04:20 dfs12 kernel: mpr0: mprsas_prepare_remove: Sending reset for target ID 34
May 21 20:04:20 dfs12 kernel: da0 at mpr0 bus 0 scbus0 target 10 lun 0
May 21 20:04:20 dfs12 kernel: da0: <ATA INTEL SSDSC2KB03 0132> s/n PHYF937402M63P8EGN detached
May 21 20:04:20 dfs12 kernel: da1 at mpr0 bus 0 scbus0 target 11 lun 0
May 21 20:04:20 dfs12 kernel: da1: <ATA INTEL SSDSC2KB03 0132> s/n PHYF937402MC3P8EGN detached
May 21 20:04:20 dfs12 kernel: da2 at mpr0 bus 0 scbus0 target 12 lun 0
May 21 20:04:20 dfs12 kernel: da2: <ATA INTEL SSDSC2KB03 0132> s/n PHYF937500NL3P8EGN detached
May 21 20:04:20 dfs12 kernel: da3 at mpr0 bus 0 scbus0 target 13 lun 0
May 21 20:04:20 dfs12 kernel: da3: <ATA INTEL SSDSC2KB03 0132> s/n PHYF937502JC3P8EGN detached
May 21 20:04:20 dfs12 kernel: da5 at mpr0 bus 0 scbus0 target 15 lun 0
May 21 20:04:20 dfs12 kernel: da5: <ATA INTEL SSDSC2KB03 0132> s/n PHYF937402P33P8EGN detached
May 21 20:04:20 dfs12 kernel: da6 at mpr0 bus 0 scbus0 target 16 lun 0
May 21 20:04:20 dfs12 kernel: da6: <ATA INTEL SSDSC2KB03 0132> s/n PHYF849300R73P8EGN detached
May 21 20:04:20 dfs12 kernel: da9 at mpr0 bus 0 scbus0 target 19 lun 0
May 21 20:04:20 dfs12 kernel: da9: <ATA INTEL SSDSC2KB03 0132> s/n PHYF937402ND3P8EGN detached
May 21 20:04:20 dfs12 kernel: ses0 at mpr0 bus 0 scbus0 target 34 lun 0
May 21 20:04:20 dfs12 kernel: ses0: <SMC SC216-P 100d> s/n Enclosure Serial Number detached
May 21 20:04:20 dfs12 kernel: da8 at mpr0 bus 0 scbus0 target 18 lun 0
May 21 20:04:20 dfs12 kernel: da8: <ATA INTEL SSDSC2KB03 0132> s/n PHYF950001GL3P8EGN detached
May 21 20:04:20 dfs12 kernel: da7 at mpr0 bus 0 scbus0 target 17 lun 0
May 21 20:04:20 dfs12 kernel: da7: <ATA INTEL SSDSC2KB03 0132> s/n PHYF937402PM3P8EGN detached
May 21 20:04:20 dfs12 kernel: (ses0:mpr0:0:34:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: da4 at mpr0 bus 0 scbus0 target 14 lun 0
May 21 20:04:20 dfs12 kernel: da4: <ATA INTEL SSDSC2KB03 0132> s/n PHYF937402MH3P8EGN detached
May 21 20:04:20 dfs12 kernel: (da5:mpr0:0:15:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: (da9:mpr0:0:19:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: (da8:mpr0:0:18:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: (da4:mpr0:0:14:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: (da3:mpr0:0:13:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: (da2:mpr0:0:12:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: (da7:mpr0:0:17:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: (da0:mpr0:0:10:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: (da6:mpr0:0:16:0): Periph destroyed
May 21 20:04:20 dfs12 kernel: (da1:mpr0:0:11:0): Periph destroyed
May 21 20:04:20 dfs12 ZFS[50490]: vdev I/O failure, zpool=$dfs12_chunk1 path=$/dev/diskid/DISK-PHYF937402M63P8EGN%20%20 offset=$270336 size=$8192 error=$6
May 21 20:04:20 dfs12 ZFS[50491]: vdev I/O failure, zpool=$dfs12_chunk1 path=$/dev/diskid/DISK-PHYF937402M63P8EGN%20%20 offset=$3840755376128 size=$8192 error=$6
May 21 20:04:20 dfs12 ZFS[50492]: vdev I/O failure, zpool=$dfs12_chunk1 path=$/dev/diskid/DISK-PHYF937402M63P8EGN%20%20 offset=$3840755638272 size=$8192 error=$6