Guide: Flashing H310/H710/H810 Mini & full size to IT Mode

[Groto]

I had no error until the last step when I enter the setsas command. I think that is different then what you experienced.

 

[BlueSandbox]

How do I enter configuration for the controller to select a boot drive? When booting there is no CTRL+C or anything about the controller

 

[fohdeesha]

@fohdeesha I tried with the latest images and get the same error. I am flashing from a custom home built PC with an Asus motherboard from about 2006. It says no LSI sas adapters found limited command set available. This happens when trying to use the setsas command after all of the flashing is completed successfully. Any ideas?

see my post on the previous page about the new images:

"Well shit, I got enough reports of this since updating the Linux ISO I'm thinking debian 11 might not work with lsiutil - can you (and anyone else with this problem including @BlueScope819 ) try starting from scratch but with this previous set of ISOs? - https://fohdeesha.com/data/other/perc/perc-crossflash-v1.8.zip"

 

[fohdeesha]

I had no error until the last step when I enter the setsas command. I think that is different then what you experienced.

if I remember right, your actual issue is the card flash is not finishing all the way (lsirec hostbnoots the card succersfully, but then the lsi-util that runs after to actually write the firmware to the card flash fails as it finds 0 lsi cards in MPT mode - the not being able to set the sas address is just a symptom of it not being flashed

@BlueScope819 had you tried the older 1.8 image since doing an nvram_clear and all that?

 

[fohdeesha]

How do I enter configuration for the controller to select a boot drive? When booting there is no CTRL+C or anything about the controller

read the section in the flashing guide about boot images

 

[fohdeesha]

OK, @BlueScope819 and anyone else having issues flashing, especially those trying to flash on non-dell hardware: I think I've gotten much closer to finding the root of the issue. It seems virtualization processor extensions (VT-d, and whatever AMD calls their version, AMD-V?) being enabled is making lsirec very unstable as it does not support IOMMU groups being present anywhere on the system. So, those having trouble, or on non-dell hardware:

Grab this debian 11 based ISO package (yes, the one that didn't work previously) - https://fohdeesha.com/data/other/perc/perc-crossflash-v1.9.zip

Now, in the BIOS, very important. Make sure all 3 of the following are DISABLED:
VT-d / processor virtualization (typically under Processor Settings)
SR-IOV Global (typically under Integrated Peripherals)
I/OAT DMA Engine (typically under Integrated Peripherals)

then go about the guide as usual, when completely done and ready to go, you can go back and enable those as needed (especially don't forget to re-enable virtualization in BIOS if you'll be running things like proxmox)

IF you still have issues flashing, and see something like "IOC failed to become ready" when running the linux flashing command, please run the following commands and post the output before rebooting: "ls -lha /sys/kernel/iommu_groups" and then "dmesg | tail"

 

[fohdeesha]

@fohdeesha I tried with the latest images and get the same error. I am flashing from a custom home built PC with an Asus motherboard from about 2006. It says no LSI sas adapters found limited command set available. This happens when trying to use the setsas command after all of the flashing is completed successfully. Any ideas?

try again following my post above using the new image package and disabling the 3 mentioned bios settings (if you can find them, you might not find the last two which is ok). I think you might be the first person to flash succesfully on a non-dell box if my theory is correct

 

[Groto]

i’m pretty sure the first time I tried this months back I disabled all those things or they didn’t exist but I will give it another shot and let you know. Appreciate the support. thanks!

 

[rojarrolla]

Hello, I have a Dell R720 Server with a Perc H710 Mini D revision card.

I started the procedure described in this link: H710 D1 Mini Blade - Fohdeesha Docs.

I ran this command: D1BLDCRS and the first time finished with errors so I ran it for a second time and it finished without issues.

But now when I try to reboot I get a message telling me:

Invalid PCIe card found in the Internal Storage slot - System Halted.

And from there I can not do anything but reboot.

I am sure that I choosed the correct procedure. Here is a photo of the cards information that I took before initiating the procedure:





Is there anything I can do to fix this? Or the card is messed up for good.

I have another one, wich shows the same information:




This allows the server to boot, and it is not flashed. So can you tell me wich could be the right procedure to flash this other card (the one with the SAS address endingo on 1700)?

Thanks a lot.

 

[Groto]

@fohdeesha No luck. The PC I am using to flash is really old with VT-x not VT-d (Core2Quad Q6600). Disabling VT-x still didn't work. It failed at setsas. I didn't see any other errors before that (only success messages) other then messages when the ISO is booting. I used the 1.9. images from your recent post. I also didn't see the other BIOS settings. Any other ideas?

 

[fohdeesha]

I am sure that I choosed the correct procedure.

You did NOT use the correct procedure - you used the Blade flashing guide, on a non-blade server. As the guide says in several places, the blade guide is only for blade servers like the M620, M820, etc. The R720 you have is a regular server, you should have followed this guide: H710 D1 Mini - Fohdeesha Docs

The adapter you already flashed blade SBRs to is bricked, you'll have to remove it and replace it to get the server to boot again

 

[fohdeesha]

@fohdeesha No luck. The PC I am using to flash is really old with VT-x not VT-d (Core2Quad Q6600). Disabling VT-x still didn't work. It failed at setsas. I didn't see any other errors before that (only success messages) other then messages when the ISO is booting. I used the 1.9. images from your recent post. I also didn't see the other BIOS settings. Any other ideas?

were you sure to reboot the server after the flashing, THEN run setsas? setsas will not work right after flashing, the server needs a reboot for the flashed firmware to properly load all the way. Can you reboot into the linux ISO again, then paste the output of the "info" command, then run the flash script again, and show the output of that as well?

 

[Groto]

were you sure to reboot the server after the flashing, THEN run setsas? setsas will not work right after flashing, the server needs a reboot for the flashed firmware to properly load all the way. Can you reboot into the linux ISO again, then paste the output of the "info" command, then run the flash script again, and show the output of that as well?

Yeah I rebooted first. Every time it fails I flash it back to default so I have something clean to start with. I’ll give it another shot as soon as I get a chance and let you know.

 

[rojarrolla]

You did NOT use the correct procedure - you used the Blade flashing guide, on a non-blade server. As the guide says in several places, the blade guide is only for blade servers like the M620, M820, etc. The R720 you have is a regular server, you should have followed this guide: H710 D1 Mini - Fohdeesha Docs

The adapter you already flashed blade SBRs to is bricked, you'll have to remove it and replace it to get the server to boot again

Then It's my mistake. Mea culpa.

Now I know that it's not a blade server.

Thanks.

 

[LayerIIX]

Edit: I just saw the post at the top of this page and think my issue might be the same. I totally didn't see that earlier even though I wanted to make sure I red through this forums first. I'm going to try that real quick.

Edit #2: So after re-reading the above posts referencing the 1.9 ISO, I reverted and then disabled the Virtualization and was able to successfully run through the process. I'll post a separate post just so @fohdeesha can have the results.

First of all, thank you for everyone that has put so much effort into this. I just purchased a Dell r720 with the intention of using it as a FreeNas Server. I'm running into a bit of a problem. I'm following the guide from H710 D1 Mini - Fohdeesha Docs. I'm able to get the part in the FreeDOS portion to process properly, however once I boot into Linux and run D1-H710, it fails. Following some others that have done this process and documented it, the noticeable difference is about half-way down where I get a message that "IOC failed to become ready. And right before there there is a fault "PTE Read access is not set".

Here's the process I've gone through and where I'm sitting at. (Again, following @fohdeesha 's guide)

Updated r720 BIOS to 2.9.0 from 2.8.0 by registering the service tag to my account and downloading the BIOS and then installing the BIOS update by launching the .efi file through the UEFI shell

I launched the FreeDOS ISO and and ran "Info" and got this:

So then I started following the H710 D1 Mini guide.

Running D1CROSS netted me this:


I don't know if this "FUSION" mode is an issue but both flashes ended with a success message.


Following the next step I rebooted into Linux and then connected via SSH from my main computer using Putty.
Running D1-H710 gets me this:


There was also this error on the console that didn't show up in the SSH client:

DMAR: DRHD: handling fault status reg 2
[ 123.598239] DMAR: [DMA Read] Request device [02:00.0] PASID ffffffff fault addr 84dfff000 [fault reason 06] PTE Read access is not set
[ 181.295500] mpt2sas_cm0: _base_spin_on_doorbell_int: failed due to timeout count (10000), int_status(c0000000)!
[ 181.295535] mpt2sas_cm0: doorbell handshake int failed (line 5934)
[ 181.295555[ mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
[ 181.295667] mpt2sas_cm0: failure at drivers/scsi/mpt3sas/mpt3sas_schih.c:11045/_scsih_probe()!

Anywho.. at this point, if I run info either on dos or on Linux, I get nothing.

I can run back into the FreeDOS and run D1REVERT and that successfully completes and allows me to start over, but I've done it a couple of different times without different results. So clearly, I'm doing something wrong or I've got bad hardware. Any ideas on what to try next would be much appreciated.

 

[LayerIIX]

OK, @BlueScope819 and anyone else having issues flashing, especially those trying to flash on non-dell hardware: I think I've gotten much closer to finding the root of the issue. It seems virtualization processor extensions (VT-d) being enabled is making lsirec very unstable as it does not support IOMMU groups being present anywhere on the system. So, those having trouble, or on non-dell hardware:

Grab this debian 11 based ISO package (yes, the one that didn't work previously) - https://fohdeesha.com/data/other/perc/perc-crossflash-v1.9.zip

Now, in the BIOS, very important. Make sure all 3 of the following are DISABLED:
VT-d / processor virtualization (typically under Processor Settings)
SR-IOV Global (typically under Integrated Peripherals)
I/OAT DMA Engine (typically under Integrated Peripherals)

then go about the guide as usual, when completely done and ready to go, you can go back and enable those as needed (especially don't forget to re-enable virtualization in BIOS if you'll be running things like proxmox)

IF you still have issues flashing, and see something like "IOC failed to become ready" when running the linux flashing command, please run the following commands and post the output before rebooting: "ls -lha /sys/kernel/iommu_groups" and then "dmesg | tail"

So I was having the issue with the "IOC failed to become ready" on the 1.9 ISO. FYI, I was able to disable the CPU Virtualization on my Dell r720 server and was able to revert and then reprocess successfully using the 1.9 ISO. I'm posting the results, but if you need more specific details, let me know and I'll get you whatever results you need from my system.




And after rebooting I was able to "setsas" and pull info.



Let me know if you need more info. I'd be more than happy to revert and rerun as a test bed as the OS Drive won't be in till this weekend.

 

[mentalinc]

@LayerIIX

What did you change between getting the error "IOC failed to become ready" and it working?
I have similar issues/errors per your screenshots with a non-dell server (AMD X570 system) trying to flash the drive.

Edit:
Of course after a few days trying, I make a post, and as soon as I make a post I figure maybe I missed something in BIOS...
So for AMD X570 Users you need to disable IOMMU. For ASUS ROG this is Advanced -> AMD CBS.

I then solved my issue the error when booting IOC " AMD-VI: Event Logged [IO_PAGE_FAULT" stopped
Card all flashed
Info now works
Windows detects card.

Thanks LayerIIX for making me realise there must be something somewhere still hiding.

@fohdeesha may be worth noting to look for "IOMMU" and disable it on AMD systems in the "Preparation" section of your wonderous guides!

I used the v1.9 if that makes a difference (I don't think it did as the error's are the same)

 

[fohdeesha]

@fohdeesha may be worth noting to look for "IOMMU" and disable it on AMD systems in the "Preparation" section of your wonderous guides!

Indeed, as my post a few replies back indicates, on the new 1.9 ISO especially, virtualization really needs to be disabled for lsirec to work reliably. Disabling IOMMU alone may work on some platforms, but disabling the root virtualization extensions themselves (AMD-v / VT-D) that create the IOMMU groups in the first place is the best course of action - I'll be adding a note to the docs guide as soon as I push a new build out (working on some other revisions at the moment as well)

 

[Groto]

@fohdeesha So I tried this in my other PC.... x370 AMD Ryzen system and disabled all of the relevant BIOS settings.

I made it all the way through to the end and when I type info it shows a SAS2308_2(D1)!!

I had one error when in the FREEDOS image. Right before it started erasing it said:

Warning! IO Base address high. Currently not supported.

But then it erased successfully..

So I rebooted into the linux image and ran through everything. Everything was success the whole time and the info command shows the LSI card with correct SAS address reprogrammmed.

1) Should I be concerned about the warning? or am I successful?
2) I think I am an idiot because the other PC I was flashing on was not connected to the internet and it seems like this time I ran the steps it downloaded an LSI image.... Is that correct? I needed an internet connection? or did something else just succeed this time where on the other PC failed?
3) When booting I used to get the PowerEdge BIOS message flash before booting into the OS... Now I don't see one. Is that normal for stock LSI? to not see a splash screen with the config or ability to enter BIOS? I realize it's no longer on Dell firmware, but I wondered if LSI had some config screen?

If I simply needed to be on the Internet/have a network connection and if that Warning! is a problem then I could try re-flashing in the original PC with a network connection this time, because I did not see the warning on the other PC.

Please confirm these questions.

Thanks again!

 

[LayerIIX]

3) When booting I used to get the PowerEdge BIOS message flash before booting into the OS... Now I don't see one. Is that normal for stock LSI? to not see a splash screen with the config or ability to enter BIOS? I realize it's no longer on Dell firmware, but I wondered if LSI had some config screen?

@Groto, Did you follow the instructions on adding the bootable Bios option? If you look at fohdeesha's guide (Introduction - Fohdeesha Docs), after you get done flashing the bios, there is an "Optional Boot Images" section. It appears by default the bootable bios is not added. I'm guessing because in the note he states that adding the bootable BIOS can add up to 2 minutes boot time depending on how many drives you have.

Edit: Ok, I'm an idiot.. you're referring to the main system bios, not the bios that lets you configure the adapter card... You can ignore me...