Guide: Flashing H310/H710/H810 Mini & full size to IT Mode

[fohdeesha]

I don’t have any PCI cards plugged in :/

that's really odd - these ISOs are built/tested on an R720. Can you ensure you're on the latest BIOS (2.9.0) and try this newer 1.9 ISO - https://fohdeesha.com/docs/store/perc/perc-crossflash-v1.9.zip

with this ISO you must ALSO disable virtualization in BIOS first, along with the other two options the guide outlines. you can turn it back on when done flashing. If you still have no luck, try a different boot method (if you're using idrac virtual media, burn a USB drive with rufus instead, or vice versa)

 

[bark_codas]

@fohdeesha thanks for posting this guide. I believe I have followed all the steps as listed but it appears to have bricked my RAID...any ideas how to fix would be great.

Server: Dell R620 1U w/PERC H710 mini
BIOS PERC H710 Package: 21.3.5-0002

Per the instructions, I unplugged the PERC battery and ensured SR-IOV Global Enable and I/OAT DMA Engine were disabled. I then proceeded with the FreeDOS instructions using https://fohdeesha.com/docs/store/perc/perc-crossflash-v1.8.zip files.

I have attached screenshots of the process:
PERC_BIOS_package - information of the PERC BIOS
FreeDOS_info - PERC information
FreeDOS_clean1 - BIOS clear pass #1
FreeDOS_clean2 - BIOS clear pass #2
RebootAfterClearing - after clearing and trying to boot into Linux OS

I am pretty sure I ended up following the wrong guide (h710 mini D1 blade) and now its bricked...

Thanks!

 

[fohdeesha]

@fohdeesha thanks for posting this guide. I believe I have followed all the steps as listed but it appears to have bricked my RAID...any ideas how to fix would be great.

Server: Dell R620 1U w/PERC H710 mini
BIOS PERC H710 Package: 21.3.5-0002

Per the instructions, I unplugged the PERC battery and ensured SR-IOV Global Enable and I/OAT DMA Engine were disabled. I then proceeded with the FreeDOS instructions using https://fohdeesha.com/docs/store/perc/perc-crossflash-v1.8.zip files.

I have attached screenshots of the process:
PERC_BIOS_package - information of the PERC BIOS
FreeDOS_info - PERC information
FreeDOS_clean1 - BIOS clear pass #1
FreeDOS_clean2 - BIOS clear pass #2
RebootAfterClearing - after clearing and trying to boot into Linux OS

I am pretty sure I ended up following the wrong guide (h710 mini D1 blade) and now its bricked...

Thanks!

You ran the blade cross flash command on a regular non-blade server, so yes that will definitely brick a card

 

[bark_codas]

Thanks for the quick response. Im assuming there is no easy way to unbrick the card? I saw a few posts on removing the card then using another desktop to boot FreeDOS and flash another firmware do you think this would be possible or its fubed and just get a replacement PERC H710 mini and pay the stupid tax? Again appreciate the assistance.

 

[fohdeesha]

Thanks for the quick response. Im assuming there is no easy way to unbrick the card? I saw a few posts on removing the card then using another desktop to boot FreeDOS and flash another firmware do you think this would be possible or its fubed and just get a replacement PERC H710 mini and pay the stupid tax? Again appreciate the assistance.

these are the perc minis which go in the proprietary small dell storage slot, you can't put them in another PC

 

[ScrumptiousTech]

@fohdeesha Great work on this guide!

I'm unable to flash my Dell PowerEdge T320's H710 B0 Full Size. Using the FreeDOS ISO's BIGB0CRS works along with resetting the card back using BIGB0RVT. The failure occurs when I boot to the Linux ISO and run B0-H710.

Bios 2.9.0
iDrac 2.65.65.65

user@debian:~$ sudo su -
root@debian:~# B0-H710
lspci: Unable to load libkmod resources: error -2
rmmod: ERROR: Module megaraid_sas is not currently loaded
rmmod: ERROR: Module mptctl is not currently loaded
rmmod: ERROR: Module mptbase is not currently loaded
Errors above are normal!
Trying unlock in MPT mode...
Device in MPT mode
Usage: /root/lsirec/lsirec <PCI ID> <operation> [args...]

PCI ID example: 0000:01:00.0

Supported operations:
  info
    Print the device state and registers
  readsbr <sbr.bin>
    Read the SBR.
  writesbr <sbr.bin>
    Write the SBR.
 *reset
    Perform a normal adapter reset. This also reloads
    the SBR.
 *halt
    Perform an HCB reset, not allowing the IOC to boot.
 *hostboot <firmware.bin>
    Reset the adapter and boot the specified firmware
    directly from host memory. Note: requires HugeTLB
    and is not compatible with IOMMUs.
 *unbind
    Unbind the kernel driver from the PCI device.
 *rescan
    Tell the kernel to remove and rescan the PCI device.
    This automatically picks up VID/PID changes and
    rebinds the driver.

* Operation forcefully unbinds the kernel driver. Make
  sure your disks are not in use!

Example: /root/lsirec/lsirec 0000:01:00.0 readsbr sbr.bin

Device in MPT mode
Usage: /root/lsirec/lsirec <PCI ID> <operation> [args...]

PCI ID example: 0000:01:00.0

Supported operations:
  info
    Print the device state and registers
  readsbr <sbr.bin>
    Read the SBR.
  writesbr <sbr.bin>
    Write the SBR.
 *reset
    Perform a normal adapter reset. This also reloads
    the SBR.
 *halt
    Perform an HCB reset, not allowing the IOC to boot.
 *hostboot <firmware.bin>
    Reset the adapter and boot the specified firmware
    directly from host memory. Note: requires HugeTLB
    and is not compatible with IOMMUs.
 *unbind
    Unbind the kernel driver from the PCI device.
 *rescan
    Tell the kernel to remove and rescan the PCI device.
    This automatically picks up VID/PID changes and
    rebinds the driver.

* Operation forcefully unbinds the kernel driver. Make
  sure your disks are not in use!

Example: /root/lsirec/lsirec 0000:01:00.0 readsbr sbr.bin

Device in MPT mode
Usage: /root/lsirec/lsirec <PCI ID> <operation> [args...]

PCI ID example: 0000:01:00.0

Supported operations:
  info
    Print the device state and registers
  readsbr <sbr.bin>
    Read the SBR.
  writesbr <sbr.bin>
    Write the SBR.
 *reset
    Perform a normal adapter reset. This also reloads
    the SBR.
 *halt
    Perform an HCB reset, not allowing the IOC to boot.
 *hostboot <firmware.bin>
    Reset the adapter and boot the specified firmware
    directly from host memory. Note: requires HugeTLB
    and is not compatible with IOMMUs.
 *unbind
    Unbind the kernel driver from the PCI device.
 *rescan
    Tell the kernel to remove and rescan the PCI device.
    This automatically picks up VID/PID changes and
    rebinds the driver.

* Operation forcefully unbinds the kernel driver. Make
  sure your disks are not in use!

Example: /root/lsirec/lsirec 0000:01:00.0 readsbr sbr.bin

Device in MPT mode
Usage: /root/lsirec/lsirec <PCI ID> <operation> [args...]

PCI ID example: 0000:01:00.0

Supported operations:
  info
    Print the device state and registers
  readsbr <sbr.bin>
    Read the SBR.
  writesbr <sbr.bin>
    Write the SBR.
 *reset
    Perform a normal adapter reset. This also reloads
    the SBR.
 *halt
    Perform an HCB reset, not allowing the IOC to boot.
 *hostboot <firmware.bin>
    Reset the adapter and boot the specified firmware
    directly from host memory. Note: requires HugeTLB
    and is not compatible with IOMMUs.
 *unbind
    Unbind the kernel driver from the PCI device.
 *rescan
    Tell the kernel to remove and rescan the PCI device.
    This automatically picks up VID/PID changes and
    rebinds the driver.

* Operation forcefully unbinds the kernel driver. Make
  sure your disks are not in use!

Example: /root/lsirec/lsirec 0000:01:00.0 readsbr sbr.bin

Pausing for 30 seconds to allow the card to boot

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014
modprobe: FATAL: Module mptctl not found in directory /lib/modules/4.9.0-12-amd64
Couldn't open /dev/mptctl or /dev/mpt2ctl or /dev/mpt3ctl!

0 MPT Ports found

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014
modprobe: FATAL: Module mptctl not found in directory /lib/modules/4.9.0-12-amd64
/bin/mknod: /dev/mptctl: File exists
Couldn't open /dev/mptctl or /dev/mpt2ctl or /dev/mpt3ctl!

0 MPT Ports found
All Done! Continue following the guide to set SAS addr
root@debian:~# r

It would be a tremendous help if you could provide any guidance.Thank you!

 

[ScrumptiousTech]

@fohdeesha Great work on this guide!

I'm unable to flash my Dell PowerEdge T320's H710 B0 Full Size. Using the FreeDOS ISO's BIGB0CRS works along with resetting the card back using BIGB0RVT. The failure occurs when I boot to the Linux ISO and run B0-H710.

Bios 2.9.0
iDrac 2.65.65.65

user@debian:~$ sudo su -
root@debian:~# B0-H710
lspci: Unable to load libkmod resources: error -2
rmmod: ERROR: Module megaraid_sas is not currently loaded
rmmod: ERROR: Module mptctl is not currently loaded
rmmod: ERROR: Module mptbase is not currently loaded
Errors above are normal!
Trying unlock in MPT mode...
Device in MPT mode
Usage: /root/lsirec/lsirec <PCI ID> <operation> [args...]

PCI ID example: 0000:01:00.0

Supported operations:
  info
    Print the device state and registers
  readsbr <sbr.bin>
    Read the SBR.
  writesbr <sbr.bin>
    Write the SBR.
*reset
    Perform a normal adapter reset. This also reloads
    the SBR.
*halt
    Perform an HCB reset, not allowing the IOC to boot.
*hostboot <firmware.bin>
    Reset the adapter and boot the specified firmware
    directly from host memory. Note: requires HugeTLB
    and is not compatible with IOMMUs.
*unbind
    Unbind the kernel driver from the PCI device.
*rescan
    Tell the kernel to remove and rescan the PCI device.
    This automatically picks up VID/PID changes and
    rebinds the driver.

* Operation forcefully unbinds the kernel driver. Make
  sure your disks are not in use!

Example: /root/lsirec/lsirec 0000:01:00.0 readsbr sbr.bin

Device in MPT mode
Usage: /root/lsirec/lsirec <PCI ID> <operation> [args...]

PCI ID example: 0000:01:00.0

Supported operations:
  info
    Print the device state and registers
  readsbr <sbr.bin>
    Read the SBR.
  writesbr <sbr.bin>
    Write the SBR.
*reset
    Perform a normal adapter reset. This also reloads
    the SBR.
*halt
    Perform an HCB reset, not allowing the IOC to boot.
*hostboot <firmware.bin>
    Reset the adapter and boot the specified firmware
    directly from host memory. Note: requires HugeTLB
    and is not compatible with IOMMUs.
*unbind
    Unbind the kernel driver from the PCI device.
*rescan
    Tell the kernel to remove and rescan the PCI device.
    This automatically picks up VID/PID changes and
    rebinds the driver.

* Operation forcefully unbinds the kernel driver. Make
  sure your disks are not in use!

Example: /root/lsirec/lsirec 0000:01:00.0 readsbr sbr.bin

Device in MPT mode
Usage: /root/lsirec/lsirec <PCI ID> <operation> [args...]

PCI ID example: 0000:01:00.0

Supported operations:
  info
    Print the device state and registers
  readsbr <sbr.bin>
    Read the SBR.
  writesbr <sbr.bin>
    Write the SBR.
*reset
    Perform a normal adapter reset. This also reloads
    the SBR.
*halt
    Perform an HCB reset, not allowing the IOC to boot.
*hostboot <firmware.bin>
    Reset the adapter and boot the specified firmware
    directly from host memory. Note: requires HugeTLB
    and is not compatible with IOMMUs.
*unbind
    Unbind the kernel driver from the PCI device.
*rescan
    Tell the kernel to remove and rescan the PCI device.
    This automatically picks up VID/PID changes and
    rebinds the driver.

* Operation forcefully unbinds the kernel driver. Make
  sure your disks are not in use!

Example: /root/lsirec/lsirec 0000:01:00.0 readsbr sbr.bin

Device in MPT mode
Usage: /root/lsirec/lsirec <PCI ID> <operation> [args...]

PCI ID example: 0000:01:00.0

Supported operations:
  info
    Print the device state and registers
  readsbr <sbr.bin>
    Read the SBR.
  writesbr <sbr.bin>
    Write the SBR.
*reset
    Perform a normal adapter reset. This also reloads
    the SBR.
*halt
    Perform an HCB reset, not allowing the IOC to boot.
*hostboot <firmware.bin>
    Reset the adapter and boot the specified firmware
    directly from host memory. Note: requires HugeTLB
    and is not compatible with IOMMUs.
*unbind
    Unbind the kernel driver from the PCI device.
*rescan
    Tell the kernel to remove and rescan the PCI device.
    This automatically picks up VID/PID changes and
    rebinds the driver.

* Operation forcefully unbinds the kernel driver. Make
  sure your disks are not in use!

Example: /root/lsirec/lsirec 0000:01:00.0 readsbr sbr.bin

Pausing for 30 seconds to allow the card to boot

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014
modprobe: FATAL: Module mptctl not found in directory /lib/modules/4.9.0-12-amd64
Couldn't open /dev/mptctl or /dev/mpt2ctl or /dev/mpt3ctl!

0 MPT Ports found

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014
modprobe: FATAL: Module mptctl not found in directory /lib/modules/4.9.0-12-amd64
/bin/mknod: /dev/mptctl: File exists
Couldn't open /dev/mptctl or /dev/mpt2ctl or /dev/mpt3ctl!

0 MPT Ports found
All Done! Continue following the guide to set SAS addr
root@debian:~# r

It would be a tremendous help if you could provide any guidance.Thank you!

SOLVED -There was another PCIE card plugged in that was causing it to not work.

 

[anarion]

Hello.

I recently got an r720xd with H710P mini D1 - I followed fohdeesha's guide and successfully flashed it to IT mode including BOOT images. I can see all disks in the controller:


I was able to install Proxmox 7.1 on that last SSD disk, but how do I now boot off of it? All I can see for booting is this:


Is there some setting I'm missing? Or is this the case of mixing SAS and SATA drives?

EDIT: Nevermind - in the contoller itself I had to mark that SSD as a boot drive first. Now all is fine

 

[fohdeesha]

in that blue menu hit alt+m for more keys, and one of them is to set a boot disk, then in the menu above that (back one level), the adapter needs to be enabled for boot as well if i remember right

 

[evilstack]

I was getting error 524288 when wiping the flash. Removing the installed Intel X520 PCIe card resolved it for me. Might be a useful addition to the preparations section of the guide.

Thanks for your work on this. I've fought with enough legacy hardware and arcane vendor utilities to assume it took some real effort.

 

[Coldstorageunit33]

Hi Everyone,

Had some issues with my card not being recognized once I reached the setsas command. Powered everything down for the night, pulled power cables, reverted to Dell flash and ran through the whole procedure again this morning and it worked like a charm. Thanks @fohdeesha !

Quick follow up question. Do I need to disable the integrated raid controller in the system bios settings?

And how does one remove the existing raid configuration from the drives so that the OS sees just the physical disk?

 

[fohdeesha]

Hi Everyone,

Had some issues with my card not being recognized once I reached the setsas command. Powered everything down for the night, pulled power cables, reverted to Dell flash and ran through the whole procedure again this morning and it worked like a charm. Thanks @fohdeesha !

Quick follow up question. Do I need to disable the integrated raid controller in the system bios settings?

And how does one remove the existing raid configuration from the drives so that the OS sees just the physical disk?

if you disable the integrated raid option in bios it will disable the integrated pcie slot and your card will no longer be visible, crossflashed or not

if you crossflashed the card correctly it's no longer a raid controller and the OS will see the individual disks, regardless of what old data is on them (the whole point of the guide)

 

[Coldstorageunit33]

if you disable the integrated raid option in bios it will disable the integrated pcie slot and your card will no longer be visible, crossflashed or not

if you crossflashed the card correctly it's no longer a raid controller and the OS will see the individual disks, regardless of what old data is on them (the whole point of the guide)

Thanks, that sorted me out!

 

[Dionysos]

Does anyone have the possibility to send me a dump of the H710 mini D1 (05CT6D I believe) SBR SPD done with lsirec or megarec?

We got an r720 server with a few spare components for our youth association and we have one H710mini D1 that is not recognized by the server (error in POST), I can reflash the card with an EPROM programmer but I don't have the right SBR.

if you have spare time to do the dump can be a huge help

many thanks in advance

 

[fohdeesha]

Does anyone have the possibility to send me a dump of the H710 mini D1 (05CT6D I believe) SBR SPD done with lsirec or megarec?

We got an r720 server with a few spare components for our youth association and we have one H710mini D1 that is not recognized by the server (error in POST), I can reflash the card with an EPROM programmer but I don't have the right SBR.

if you have spare time to do the dump can be a huge help

many thanks in advance

the SBR file you need is in the freeDOS iso in my guide, or you can grab it directly from the sources on my github:

Modified for IT mode: lab-docu/D1Mod.sbr at master · Fohdeesha/lab-docu

Original stock: lab-docu/D1Stock.sbr at master · Fohdeesha/lab-docu

 

[Dionysos]

the SBR file you need is in the freeDOS iso in my guide, or you can grab it directly from the sources on my github:

Modified for IT mode: lab-docu/D1Mod.sbr at master · Fohdeesha/lab-docu

Original stock: lab-docu/D1Stock.sbr at master · Fohdeesha/lab-docu

Much appreciated

 

[z00100]

Hi @fohdeesha ,

Thank you so much for your work and this is absolutely fantastic.

I have a really dumb question but I can't seem to find the answer for it. I just got a DELL R620 with a PERC710p mini card. Right now I have the system set up with 4 SAS drives running in RAID 10 with Proxmox. I have a fair bit of VM's and containers etc installed on it.

Will crossflashing the PERC controller to IT mode nuke my RAID 10 Config? Will installing the optional boot images (both BIOS and UEFI) keep the existing RAID configuration?

Thanks for the help.

 

[bonox]

yes, it will destroy your data. Or more properly, your ability to see that data, which is pretty much the same thing.

HBAs are not RAID. RAID presents imaginary storage volumes to the OS; you could have a single drive presented to the OS that is actually made of of twenty physical disks and the OS wouldn't have a clue about what lies beneath. An HBA presents the real physical disks themselves to the OS. Remove the RAID LUN and proxmox won't know what just happened. And if you used to boot off that volume, it won't any more.

IT Mode is an HBA.

Removing the RAID config through the controller (which is what will happen when you replace the firmware) is pretty much the same as just pulling the drives completely and then plugging in different drives hanging off a different controller.

 

[Zilch]

Tried following the guide to a T about a dozen times for the H710 Mini D1, but keep getting errors when I get to setting the sas address.

No other PCI cards installed and tried powering down and unplugging for 30 minutes several times. Also tried it on a H710P card with that specific guide, but ran into the same issue.

I've been doing everything through iDRAC, any chance that would be an issue? Does it matter if drives are installed? I'm at a loss for what to try next.

Any help or guidance would be greatly appreciated!

 

[Zilch]

Disregard. Virtualization wasn't turned off. Make sure you fully read the intro guide too folks!