Guide: Flashing H310/H710/H810 Mini & full size to IT Mode

[amp88]

The 122 "Device Error Count" value in conjunction with the details of the four errors included in that log (WRITE FPDMA QUEUED) are concerning.

Could you also post the output from this command, please (it might help give more information about the root cause):

grep -E "(mpt[23]sas| scsi | sd )" /var/log/messages

This command comes from Art of Server's video on analysing log files to help troubleshoot/diagnose LSI controller/drive problems.

 

[josh]

The 122 "Device Error Count" value in conjunction with the details of the four errors included in that log (WRITE FPDMA QUEUED) are concerning.

Could you also post the output from this command, please (it might help give more information about the root cause):

grep -E "(mpt[23]sas| scsi | sd )" /var/log/messages

This command comes from Art of Server's video on analysing log files to help troubleshoot/diagnose LSI controller/drive problems.

Deleted the pool and recreated it and the write errors have clogged up again and tripped the FAULT threshold on everything. No read errors, only writes. Something's up with the controller.

Nov 28 14:06:05 ba1 kernel: [ 3616.006258] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:05 ba1 kernel: [ 3616.006275] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:05 ba1 kernel: [ 3616.006326] sd 0:0:1:0: [sdb] tag#1879 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:05 ba1 kernel: [ 3616.006338] sd 0:0:1:0: [sdb] tag#1879 CDB: Write(16) 8a 00 00 00 00 05 74 ff b6 20 00 00 00 e0 00 00
Nov 28 14:06:06 ba1 kernel: [ 3616.756069] sd 0:0:1:0: Power-on or device reset occurred
Nov 28 14:06:06 ba1 kernel: [ 3617.256268] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:07 ba1 kernel: [ 3618.006040] sd 0:0:1:0: Power-on or device reset occurred
Nov 28 14:06:08 ba1 kernel: [ 3618.506217] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:08 ba1 kernel: [ 3619.256032] sd 0:0:2:0: Power-on or device reset occurred
Nov 28 14:06:09 ba1 kernel: [ 3619.756192] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:09 ba1 kernel: [ 3619.756206] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:09 ba1 kernel: [ 3619.756213] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:10 ba1 kernel: [ 3620.505996] sd 0:0:3:0: Power-on or device reset occurred
Nov 28 14:06:10 ba1 kernel: [ 3621.006149] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:10 ba1 kernel: [ 3621.006167] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:10 ba1 kernel: [ 3621.006202] sd 0:0:3:0: [sdd] tag#1888 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:10 ba1 kernel: [ 3621.006208] sd 0:0:3:0: [sdd] tag#1859 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:10 ba1 kernel: [ 3621.006214] sd 0:0:3:0: [sdd] tag#1888 CDB: Read(16) 88 00 00 00 00 05 74 ff ff f0 00 00 00 08 00 00
Nov 28 14:06:10 ba1 kernel: [ 3621.006223] sd 0:0:3:0: [sdd] tag#1859 CDB: Write(16) 8a 00 00 00 00 00 00 80 28 30 00 00 00 08 00 00
Nov 28 14:06:11 ba1 kernel: [ 3621.755948] sd 0:0:3:0: Power-on or device reset occurred
Nov 28 14:06:11 ba1 kernel: [ 3622.256195] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:11 ba1 kernel: [ 3622.256211] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:11 ba1 kernel: [ 3622.256248] sd 0:0:0:0: [sda] tag#1858 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:11 ba1 kernel: [ 3622.256264] sd 0:0:0:0: [sda] tag#1858 CDB: Write(16) 8a 00 00 00 00 00 03 00 28 40 00 00 00 08 00 00
Nov 28 14:06:11 ba1 kernel: [ 3622.258279] sd 0:0:0:0: [sda] tag#1917 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:11 ba1 kernel: [ 3622.258285] sd 0:0:0:0: [sda] tag#1917 CDB: Write(16) 8a 00 00 00 00 00 03 00 28 38 00 00 00 08 00 00
Nov 28 14:06:12 ba1 kernel: [ 3623.005960] sd 0:0:0:0: Power-on or device reset occurred
Nov 28 14:06:18 ba1 kernel: [ 3628.756035] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:18 ba1 kernel: [ 3628.756052] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:18 ba1 kernel: [ 3628.756088] sd 0:0:2:0: [sdc] tag#1879 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:18 ba1 kernel: [ 3628.756105] sd 0:0:2:0: [sdc] tag#1879 CDB: Write(16) 8a 00 00 00 00 00 02 80 28 50 00 00 00 08 00 00
Nov 28 14:06:18 ba1 kernel: [ 3628.758103] sd 0:0:2:0: [sdc] tag#1875 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:18 ba1 kernel: [ 3628.758109] sd 0:0:2:0: [sdc] tag#1875 CDB: Write(16) 8a 00 00 00 00 00 01 00 28 40 00 00 00 08 00 00
Nov 28 14:06:19 ba1 kernel: [ 3629.505812] sd 0:0:2:0: Power-on or device reset occurred
Nov 28 14:06:34 ba1 kernel: [ 3645.005766] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:34 ba1 kernel: [ 3645.005783] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:34 ba1 kernel: [ 3645.005825] sd 0:0:3:0: [sdd] tag#1866 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:34 ba1 kernel: [ 3645.005836] sd 0:0:3:0: [sdd] tag#1866 CDB: Write(16) 8a 00 00 00 00 00 02 80 28 60 00 00 00 08 00 00
Nov 28 14:06:34 ba1 kernel: [ 3645.007973] sd 0:0:3:0: [sdd] tag#1865 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:34 ba1 kernel: [ 3645.007981] sd 0:0:3:0: [sdd] tag#1865 CDB: Write(16) 8a 00 00 00 00 00 02 00 28 30 00 00 00 08 00 00
Nov 28 14:06:35 ba1 kernel: [ 3645.755535] sd 0:0:3:0: Power-on or device reset occurred
Nov 28 14:06:35 ba1 kernel: [ 3646.255784] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:35 ba1 kernel: [ 3646.255799] mpt2sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Nov 28 14:06:35 ba1 kernel: [ 3646.255840] sd 0:0:1:0: [sdb] tag#1919 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:35 ba1 kernel: [ 3646.255857] sd 0:0:1:0: [sdb] tag#1919 CDB: Write(16) 8a 00 00 00 00 05 74 ff b6 20 00 00 00 e0 00 00
Nov 28 14:06:35 ba1 kernel: [ 3646.257922] sd 0:0:1:0: [sdb] tag#1918 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK cmd_age=0s
Nov 28 14:06:35 ba1 kernel: [ 3646.257929] sd 0:0:1:0: [sdb] tag#1918 CDB: Write(16) 8a 00 00 00 00 00 00 00 0a 20 00 00 00 e0 00 00
Nov 28 14:06:36 ba1 kernel: [ 3647.005554] sd 0:0:1:0: Power-on or device reset occurred

 

[amp88]

Deleted the pool and recreated it and the write errors have clogged up again and tripped the FAULT threshold on everything. No read errors, only writes. Something's up with the controller.

I just noticed that the "UDMA_CRC_Error_Count" value from the original smartctl output is also showing 122. Is there any chance that any of the cables in your system (e.g. from the backplane to the controller) aren't properly plugged in all the way/seated correctly? Did you use this server before flashing the controller, and if so, did you have storage problems then too?

 

[josh]

I just noticed that the "UDMA_CRC_Error_Count" value from the original smartctl output is also showing 122. Is there any chance that any of the cables in your system (e.g. from the backplane to the controller) aren't properly plugged in all the way/seated correctly? Did you use this server before flashing the controller, and if so, did you have storage problems then too?

Not much sample data from before the flashing. The drives don't appear at all if I remove the controller so I have no idea how to test if it's a controller issue or a backplane issue.

 

[amp88]

Not much sample data from before the flashing. The drives don't appear at all if I remove the controller so I have no idea how to test if it's a controller issue or a backplane issue.

I'm just getting ready to go, so hopefully someone else can help you out. I'd maybe try moving the drives to other bays to see if the errors stop or persist, but I don't know if that's feasible with your setup.

 

[josh]

I'm just getting ready to go, so hopefully someone else can help you out. I'd maybe try moving the drives to other bays to see if the errors stop or persist, but I don't know if that's feasible with your setup.

I've just tried reseating all the BP cables and shifting some drives to the 6 bays powered by the second BP cable. System is rebooting but don't have any expectations.

Edit: Same issues

 

[bmxxmikexx]

I have an R720 + H710 mini B0 and seem to be getting tripped up with something in the flashing process (after B0-H710). I have all 3 options disabled in the bios. The battery is sitting on my desk. I have reverted to B0REVERT and have cleaned many times with B0CROSS for safe measures.

Edit: also pulled risers because I don't see any other pcie cards.




A bit stuck so I could certainly use a sanity check.

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

0 MPT Ports found
All Done! Continue following the guide to set SAS addr
root@debian:~#
root@debian:~#
root@debian:~# B0-H710
rmmod: ERROR: Module megaraid_sas is not currently loaded
Errors above are normal!
Trying unlock in MPT mode...
Device in MPT mode
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Setting up HCB...
HCDW virtual: 0x7fd3a7800000
HCDW physical: 0x123400000
Loading firmware...
Loaded 809340 bytes
Booting IOC...
IOC is READY
IOC Host Boot successful.
Device in MPT mode
Removing PCI device...
Rescanning PCI bus...
PCI bus rescan complete.
Pausing for 30 seconds to allow the card to boot

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

1 MPT Port found

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC
1.  ioc0              LSI Logic SAS2308 B0      200      14000700     0

1.  NVSRAM
2.  SEEPROM
3.  FLASH
4.  BootLoader
5.  Firmware (backup copy)
6.  Firmware (current copy)
7.  Persistent non-manufacturing config pages
8.  Persistent manufacturing config pages
9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit] 3

Erasing...

1.  NVSRAM
2.  SEEPROM
3.  FLASH
4.  BootLoader
5.  Firmware (backup copy)
6.  Firmware (current copy)
7.  Persistent non-manufacturing config pages
8.  Persistent manufacturing config pages
9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit] 8

Erasing...

Failed to issue command

Erase failed

1.  NVSRAM
2.  SEEPROM
3.  FLASH
4.  BootLoader
5.  Firmware (backup copy)
6.  Firmware (current copy)
7.  Persistent non-manufacturing config pages
8.  Persistent manufacturing config pages
9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit]

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

0 MPT Ports found
All Done! Continue following the guide to set SAS addr
root@debian:~#

 

[fohdeesha]

I have an R720 + H710 mini B0 and seem to be getting tripped up with something in the flashing process (after B0-H710). I have all 3 options disabled in the bios. The battery is sitting on my desk. I have reverted to B0REVERT and have cleaned many times with B0CROSS for safe measures.

Edit: also pulled risers because I don't see any other pcie cards.


View attachment 26383

A bit stuck so I could certainly use a sanity check.

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

0 MPT Ports found
All Done! Continue following the guide to set SAS addr
root@debian:~#
root@debian:~#
root@debian:~# B0-H710
rmmod: ERROR: Module megaraid_sas is not currently loaded
Errors above are normal!
Trying unlock in MPT mode...
Device in MPT mode
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Setting up HCB...
HCDW virtual: 0x7fd3a7800000
HCDW physical: 0x123400000
Loading firmware...
Loaded 809340 bytes
Booting IOC...
IOC is READY
IOC Host Boot successful.
Device in MPT mode
Removing PCI device...
Rescanning PCI bus...
PCI bus rescan complete.
Pausing for 30 seconds to allow the card to boot

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

1 MPT Port found

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC
1.  ioc0              LSI Logic SAS2308 B0      200      14000700     0

1.  NVSRAM
2.  SEEPROM
3.  FLASH
4.  BootLoader
5.  Firmware (backup copy)
6.  Firmware (current copy)
7.  Persistent non-manufacturing config pages
8.  Persistent manufacturing config pages
9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit] 3

Erasing...

1.  NVSRAM
2.  SEEPROM
3.  FLASH
4.  BootLoader
5.  Firmware (backup copy)
6.  Firmware (current copy)
7.  Persistent non-manufacturing config pages
8.  Persistent manufacturing config pages
9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit] 8

Erasing...

Failed to issue command

Erase failed

1.  NVSRAM
2.  SEEPROM
3.  FLASH
4.  BootLoader
5.  Firmware (backup copy)
6.  Firmware (current copy)
7.  Persistent non-manufacturing config pages
8.  Persistent manufacturing config pages
9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit]

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

0 MPT Ports found
All Done! Continue following the guide to set SAS addr
root@debian:~#

View attachment 26384

remove any hard drives in the server, cold power cycle it, then try again starting from the linux ISO

 

[bmxxmikexx]

remove any hard drives in the server, cold power cycle it, then try again starting from the linux ISO

It worked! I am honored that you took the time to help me. Thank you so, so much for what you have done for this community. My passion for homelabbing has shot through the roof between your brocade guide and this one. Truly, thank you. I hope you have a wonderful new year.

 

[Coltonc18]

Hello,

I would really appreciate some assistance here, I am trying to follow the guide to flash my raid, and am having no luck.

I originally started with FreeDos, but when I do it doesn't run at all. Like I have it in the USB and I turn on the server, it goes through inilitization, and then nothing happens.

When I try the Debian, it does run and pull up, but when I do Sudo info, it tells me "no lsi sas adapters found" I know there is a LSI adapter as I can see it in my settings, and I pulled the battery off of it.

I have tried to go in through iDrac so I could just do it through there, but for some reason I can never get my iDrac to pull up, I show it on my network, but it always says connection refused, so I cant go that route.

Any help would be greatly appreciated.

 

[Coltonc18]

To clarify, I have double and triple checked, nothing else installed and everything turned off in BIOS

 

[fohdeesha]

Hello,

I would really appreciate some assistance here, I am trying to follow the guide to flash my raid, and am having no luck.

I originally started with FreeDos, but when I do it doesn't run at all. Like I have it in the USB and I turn on the server, it goes through inilitization, and then nothing happens.

When I try the Debian, it does run and pull up, but when I do Sudo info, it tells me "no lsi sas adapters found" I know there is a LSI adapter as I can see it in my settings, and I pulled the battery off of it.

I have tried to go in through iDrac so I could just do it through there, but for some reason I can never get my iDrac to pull up, I show it on my network, but it always says connection refused, so I cant go that route.

Any help would be greatly appreciated.

the info command in linux is looking for IT mode adapters only, so if you've skipped the freedos flashing step, it's normal that it won't show up. You need to do the freedos flashing steps first. You've followed *everything* on the intro page, turning off all mentioned bios settings? and you're 100% sure the server is set to BIOS boot mode, not UEFI? If you still can't get it to boot the freedos ISO, re-download the zip and try again, perhaps your download got corrupted. But if you don't even see it attempt to boot freedos, that usually means the server is set for uefi boot. How are you burning the freedos ISO to a usb drive? what program?

 

[Coltonc18]

the info command in linux is looking for IT mode adapters only, so if you've skipped the freedos flashing step, it's normal that it won't show up. You need to do the freedos flashing steps first. You've followed *everything* on the intro page, turning off all mentioned bios settings? and you're 100% sure the server is set to BIOS boot mode, not UEFI? If you still can't get it to boot the freedos ISO, re-download the zip and try again, perhaps your download got corrupted. But if you don't even see it attempt to boot freedos, that usually means the server is set for uefi boot. How are you burning the freedos ISO to a usb drive? what program?

I couldn't get it to boot freedos at all. The only way I got it to boot was to try the second iso you had in the file Debian. But that is where it wasn't finding my raid card.

Its set to boot on BIOS, and all 3 settings are changed in bios.

I will go and download it again and try to put it on a flash drive again.

I was using Rufus to do it. Writing it using the DD method.

Thanks, I'll update after I try it with a new download.

 

[fohdeesha]

I couldn't get it to boot freedos at all. The only way I got it to boot was to try the second iso you had in the file Debian. But that is where it wasn't finding my raid card.

Its set to boot on BIOS, and all 3 settings are changed in bios.

I will go and download it again and try to put it on a flash drive again.

I was using Rufus to do it. Writing it using the DD method.

Thanks, I'll update after I try it with a new download.

definitely use rufus to write the iso to a drive using the default rufus iso method, not DD. when the server boots smash f11 (or is it f12? can't remember) to get into the boot selection menu, and choose the usb drive (should be like cdrom: usbdrivename or something like that)

 

[Coltonc18]

definitely use rufus to write the iso to a drive using the default rufus iso method, not DD. when the server boots smash f11 (or is it f12? can't remember) to get into the boot selection menu, and choose the usb drive (should be like cdrom: usbdrivename or something like that)

Oh it says to use the dd method in the guide? I'll change to iso and do the f11

 

[fohdeesha]

Oh it says to use the dd method in the guide? I'll change to iso and do the f11

it's meant to say you can use rufus, *or* use dd in linux, but not using dd method within rufus (although they should end up identical, but who knows). it could definitely use rewording. booting dos via iso is finnicky

 

[Coltonc18]

Ugh dang files too big, trying to show a pic.... I got to boot menu, and clicked on bios boot menu... Now no signal again.

 

[Coltonc18]

There was a normal boot, bios boot menu, UEFI boot menu, driver health menu, launch system setup, and system utilities.

 

[fohdeesha]

There was a normal boot, bios boot menu, UEFI boot menu, driver health menu, launch system setup, and system utilities.

did you choose bios boot menu then select the flash drive/cdrom?

 

[fohdeesha]

updated the guide to finally move the H310 SBR flashing steps to freedos, to match the other models. All card models now flash the same way, first in freedos to wipe the card and flash the appropriate sbr, then in linux to flash the firmware file

also added note to intro page that hard drives must be removed first (a few reports of drives being plugged in causing the firmware flash to fail inside linux). New ISOs built and the linux ISO is built on a newer debian so hopefully continuing to reduce the amount of compatibility/flash issues