Help please configuring a HBA (LSI 9212 4i4e). Odd boot and bios issues.

[slightlyconfused]

Hi,

I recently posted a number of questions regarding SAS adaptors. To save you reading them, I’m looking for one that would add more sata drives and give me access to an LTO tape drive.

My initial foray into the world of IT HBAs was not great and I ended up returning the 1st card I purchased. I now have a cheaper, in that if it does not work its not a great loss, but older IBM branded LSI 9212 4i4e. This is a bit of an odd card in that it has 4 sata internal ports, rather than a single sas and then 1 external sas port. Its also only a pcie 2.0 card.

I chose this as it has the external port I need for the LTO so will not need an internal to external adapter card and I can just plug in whatever sata drive I need, making the install a little more tidy.

However as you will probably guess. I’m having a few problems, listed I order of annoyance.

1. If the card is inserted and the PC bios is set to UFEI then I cannot access the PC BIOS at all!
If the PC BIOS is set to boot in legacy mode then I can access its bios after the HBA has initialised.
Drives connected to the card are recognised in Linux whether I boot in legacy or uefi mode.

2. The card is much slower than I anticipated it would be. With one SSD connected I am getting around 130mbs, for comparison an SSD plugged into the PCs sata ports does around 515mbs.


These issues might be due to the HBA’s bios, which is 10.00.05-IT. I need it in IT mode so that’s fine, but looking on Broadcom’s web site ther appear to be later versions.

Support Documents and Downloads=

I confess I cannot make head or tail of broadcom’s version control here. There is an UEFI_BSD 20 with version of 7.27.01.01 and a SAS2UEFI_BSD_P10 with a version of 07.17.00.04

There are then a set of installers none of which include the firmware, with the exception of the windows one which does and is 20.00.07.00!


So questions.

1. If I take the sas2flash rel package (I’m using Linux mint) can I use that to flash the firmware that is in the Windows zip?

2. What are the chances of a bad flash bricking the card, that is somehow putting the wrong software on it. Thinking more: can I flash an IBM branded card with broadcom's firmware? Reading around there appears to be an issue cross flashing to Del branded cards, though I may have missunderstood here.

3. What are those UEFI_BSD 20 and SAS2UEFI_BSD_P10 files anyway?

4. Assuming that the latest firmare is the one included in the windows zip. Do you think flashing to it will make the card run faster and fix the odd bios access issue I am having? I appreicate the answer may be try it and see.

Massive thanks in advance.

 

[slightlyconfused]

Hello (again),

Since the above post I have been reading though the quick guide (from the above link) regarding sas to flash and found the following from the HBA:

LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

Adapter Selected is a LSI SAS: SAS2008(B2)

<snip>
NVDATA Version (Default) : 0a.02.00.16
NVDATA Version (Persistent) : 0a.02.00.16
Firmware Product ID : 0x2213 (IT)
Firmware Version : 10.00.05.00
NVDATA Vendor : LSI
NVDATA Product ID : Undefined
BIOS Version : 07.19.00.00
UEFI BSD Version : 07.18.01.05
FCODE Version : N/A
Board Name : IBM 6Gb SAS HBA
<snip>

Finished Processing Commands Successfully.
Exiting SAS2Flash.

The version numbers on the linked page now make some sort of sense, except that the latest UEFI BSD on the Broadcom site is 07.17.00.04, and my HBA has 07.18.10.05. Any idea how this is possible or where the version on the HBA came from?

Having backed up (uploaded from HBA to pc) the HBA's firmware and bios I have hit a second issue with the bios.

The one in the linked (above) windows package is 202,240 bytes, but my back up is 407,040 bytes. Shouldn't the new file be at least the same size if not larger? The quick guide to SAS2flash states that backing up the bios also copies the UFEI and fcode) but does not tell you how to split the bios out, or just backup the bios.

The firmware file sizes make more sense, with the backup being 662,420 bytes and the linked file (214i42t.bin) at 722,708 bytes so I have flashed that to the card and just need to speed test. I have also backed up the whole flash just in case.

I still cannot acces the PCs bios with the card inserted. I just get a black screen.

Any help appreciated.

 

[slightlyconfused]

Just a quick update, unless the newer bios has any effect on speed, which I suspect it will not, this card is a dud. The speed test came back slightly slower than before, around 114mbs. The same ssd runs at 520mbs when plugged directly into a mbo sata port. I still have no idea why it is preventing the PC from entering its (the PC's) bios. A shame, but it would cost me more to send back than the card cost.

You live and learn.

 

[luckylinux]

Well lots of Questions and I don't think I can answer all of them, far from it.

a. Which Motherboard are you using ? Did you check if BIOS has been updated to latest ?

b. In which PCIe Slot did you plug in the Card? Is it directly connected to CPU or is it PCH/DMI/Chipset connected PCIe Slot ? If PCH/DMI/Chipset it might only be x4 GEN2, but that should still be plenty (2000 MB/s).

c. Did you enable "Above 4G Decoding" and "Re-sizeable Bar" Options in BIOS ? Cannot remember exactly which one but if you have lots of RAM installed, there can be some "overlap" with the PCIe Address Space.

d. Did you try to clean flash the HBA (remove EVERYTHING form the HBA), then ONLY install the Firmware in IT Mode, WITHOUT BIOS/EFI ? Does it still give Issues, if there is no BIOS/UEFI on it ?

e. Did you try disabling "PCIe OPROM Boot" for the affected PCIe Slot ? Or set it to Legacy Only. If it's trying to load a UEFI Firmware which does not exist on the HBA or if it's not signed and you have Secureboot enabled, then it might cause Issues

f. The UEFI and SAS2UEFI are several Things .... In one Archive, there is the Signed (Secureboot from 10 Years ago ...) UEFI Bootloader, the Unsigned (from 10 Years ago ...) UEFI Bootloader and the flashing Tool sas2flash.efi or possibly sas2flsh.efi (although I think it's the Former). I cannot remember the Naming Scheme, their thing is totally messed up IMHO. Also sas2flsh.exe (DOS/FreeODS) and sas2flash.exe (Windows)

What I typically do is press several Times F11 (Select Boot Option Menu) on Supermicro Motherboards during Post and select UEFI Shell. If you don't have that Option, maybe go into BIOS and see in the Boot Section if you can select it. Otherwise you might need a USB Flashdrive to launch UEFI Shell. Example on how to do that: GitHub - luckylinux/aspm-troubleshooting: aspm-troubleshooting

 

[slightlyconfused]

Hi, many thanks for the suggestions. I've marked though them with where I am right now. There is a lot to check on here.

Well lots of Questions and I don't think I can answer all of them, far from it.

a. Which Motherboard are you using ? Did you check if BIOS has been updated to latest ?

Gigabyte Aorus x570 Elite and yes latest bios.

b. In which PCIe Slot did you plug in the Card? Is it directly connected to CPU or is it PCH/DMI/Chipset connected PCIe Slot ? If PCH/DMI/Chipset it might only be x4 GEN2, but that should still be plenty (2000 MB/s).

Good point, I hadn't thought of that and it may be part of the issue. The HBA is in the second slot and on checking this is pcie v4.0 but only 4 lanes, so yes the max would be 2k mb/s. As you say it should be enough and the 114mbs I am getting is waay down on what I expected.

c. Did you enable "Above 4G Decoding" and "Re-sizeable Bar" Options in BIOS ? Cannot remember exactly which one but if you have lots of RAM installed, there can be some "overlap" with the PCIe Address Space.

No and no, I have 32GB ram, so will check on this
<edit> have now checked I have both options disabled as of now. </edit>

d. Did you try to clean flash the HBA (remove EVERYTHING form the HBA), then ONLY install the Firmware in IT Mode, WITHOUT BIOS/EFI ? Does it still give Issues, if there is no BIOS/UEFI on it ?

No I just did an update as I did not know that this would be an issue. I also did not know you could run these cards without a bios so I have not checked that.
Does anyone know why my bios is a different version to the one on the Broadcom site?

e. Did you try disabling "PCIe OPROM Boot" for the affected PCIe Slot ? Or set it to Legacy Only. If it's trying to load a UEFI Firmware which does not exist on the HBA or if it's not signed and you have Secureboot enabled, then it might cause Issues

I don't have an per slot or indeed any option to disable OPROM Boot. I can however set boot for storage devices to either UFEI or lagacy, which I assumed was the mbo sata and nvme connections, or set PCIE to ufei or legacy. As mentioned in my 1st post, if I set storage to legacy then the card bios loads and I can access the PC one. With the PC bios set to book ufei then I cannot access the PC bios. I confess I never tried setting just pcie to legacy. I'll need to check this.

f. The UEFI and SAS2UEFI are several Things .... In one Archive, there is the Signed (Secureboot from 10 Years ago ...) UEFI Bootloader, the Unsigned (from 10 Years ago ...) UEFI Bootloader and the flashing Tool sas2flash.efi or possibly sas2flsh.efi (although I think it's the Former). I cannot remember the Naming Scheme, their thing is totally messed up IMHO. Also sas2flsh.exe (DOS/FreeODS) and sas2flash.exe (Windows)

What I typically do is press several Times F11 (Select Boot Option Menu) on Supermicro Motherboards during Post and select UEFI Shell. If you don't have that Option, maybe go into BIOS and see in the Boot Section if you can select it. Otherwise you might need a USB Flashdrive to launch UEFI Shell. Example on how to do that: GitHub - luckylinux/aspm-troubleshooting: aspm-troubleshooting

Once again many thanks, I need to do some checking.

 

[luckylinux]

Good point, I hadn't thought of that and it may be part of the issue. The HBA is in the second slot and on checking this is pcie v4.0 but only 4 lanes, so yes the max would be 2k mb/s. As you say it should be enough and the 114mbs I am getting is waay down on what I expected.

Well throughput and IOPS are different Things and the LSI 9211-8i (PCIe 2.0) is struggling with several SSDs. With a couple or even 4, it should be able to handle it just fine though. LSI 9207-8i Types (PCIe 3.0) are a bit better in that Regards.

No and no, I have 32GB ram, so will check on this
<edit> have now checked I have both options disabled as of now. </edit>

And why don't you enable those ?

No I just did an update as I did not know that this would be an issue. I also did not know you could run these cards without a bios so I have not checked that.
Does anyone know why my bios is a different version to the one on the Broadcom site?

Your BIOS or your Firmware ?

It could be just the OEM who customized their Firmware / BIOS. Dell typically does that. I think IBM did the same ...

I don't have an per slot or indeed any option to disable OPROM Boot. I can however set boot for storage devices to either UFEI or lagacy, which I assumed was the mbo sata and nvme connections, or set PCIE to ufei or legacy. As mentioned in my 1st post, if I set storage to legacy then the card bios loads and I can access the PC one. With the PC bios set to book ufei then I cannot access the PC bios. I confess I never tried setting just pcie to legacy. I'll need to check this.

Yeah there is some incompatibility going on.

Is Secureboot enabled ?

Once again many thanks, I need to do some checking.

Good Luck .

Another possibility is Cooling: did you check that the HBA (and/or possibly your SSDs) are getting sufficient Airflow ???
They might be running too hot and throttling.

EDIT 1: if you want any help it's probably easier to post some Screenshots from UEFI Shell and the output of sas2flash.efi -listall and sas2flash.efi -c 0 -list

You seem also to have quite a bit of confusion about what exists in Terms of Files to Download or on the HBA Card itself.

On the HBA itself:
- Firmware (IT or IR Mode)
- (optional) BIOS Bootloader (to be able to boot from Disks attached to the HBA, in Legacy/CSM/BIOS Mode)
- (optional) UEFI Bootloader (to be able to boot from Disks attached to the HBA, in UEFI Mode)

If you want to cross-flash to generic LSI Firmware, you will probably also need to clean the Card with megarec and upload a sbrempty.bin File (from FreeDOS). There are like 1000 Tutorials online, mostly focused on the LSI 9211-8i, but the Procedure should be the same for all.

In Terms of Downloads there is quite a bit more:
- Tools to flash/get information from UEFI Shell (sas2flash.efi)
- Tools to flash/get information from [Free]DOS/Windows (sas2flsh.exe and sas2flash.exe respectively)
- Tools to flash/get information from Linux
- Tools to flash/get information from Solaris
- Tools to flash/get information from ...

And like 4 or 5 versions for each ...

I do NOT reccomend performing Operations on the Card from Linux/Solaris/etc. It's much better to have a Minimal System (I reccomend UEFI) or potentially FreeDOS (although, on new Systems, that can sometimes trigger a Failed to Initialize PAL Error Message, thus I reccomend UEFI)

 

[slightlyconfused]

Thanks for the reply. When I left things last night I had flashed the HBA firmware, but not the bios or ufei as I needed to sleep on the version issue. Since its an IBM branded card I suspect you are correct regading IBM doing something 'odd' and this would explain the different versions. I left the PC bios set to UFEI and the card could be seen by sas2flash.

This morning....

Sas2flash cannot see the card! I get.

LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

No LSI SAS adapters found! Limited Command Set Available!
ERROR: Command Not allowed without an adapter!
ERROR: Couldn't Create Command -list
Exiting Program.

Here's the output from sas2flash from my second post in this thread, showing that it could see the card yesterday.

LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

Adapter Selected is a LSI SAS: SAS2008(B2)

<snip>
NVDATA Version (Default) : 0a.02.00.16
NVDATA Version (Persistent) : 0a.02.00.16
Firmware Product ID : 0x2213 (IT)
Firmware Version : 10.00.05.00
NVDATA Vendor : LSI
NVDATA Product ID : Undefined
BIOS Version : 07.19.00.00
UEFI BSD Version : 07.18.01.05
FCODE Version : N/A
Board Name : IBM 6Gb SAS HBA
<snip>

Finished Processing Commands Successfully.
Exiting SAS2Flash.

Yet lspci can see the card. If I switch the PC bios to legacy mode the card bios loads and I can enter the HB config with ctrl-c. Like huh?

I grabbed my old windows disk, booted it and tried the .exe version of sas2flash. Same result it cannot see the card. Sadly I cannot try the card in another pc as I don't have one.

Taking a step back I set up a freedos usb using rufus and tried the dos version of sas2flsh. As you suspected it errors out with Failed to Initialise Pal.

I am now very stuck. The PC mbo is not a server, its a standard highish end consumer and to be honest its bios is very very limited. I have tried following a number of guides to get an efi shell to load, but while I can see the option the the boot menu, selecting it just takes me back to the boot menu.

Booting into freedos is obviously a nonstarter as it errors with the Failed to Initialise Pal. I have a feeling that my mbo is going to be a real pain to get to work with an efi shell, though any suggestions on that would be great.

I am also baffled as to why the card was visable to flash2sas last night and now is not.

<edit 14:41 on 21 Apr 25>
After more checking I have confirmed that this is not an issue where I flashed the unsigned firmware but have tried to boot using secure boot. Secure boot is off.
I also discovered that LSI used to provide, or were thinking of providing, preboot software > LSI Pre-Boot USB tool Download < .However the links are ftp only (so modern browsers just shrug at them). Filezilla can log into the ftp server, but just shows empty directories.

Does anyone know if LSI/broadcom etc actually provided these files?
</edit>

 

[slightlyconfused]

Hello,

A lot of progress since my last post. Firstly I have managed to boot into a working efi shell. It appears that my mbo is very fussy and while I had named the shell correctly on the freedos usb I had the wrong one. The shell linked indirectly from this Redit post > Reddit post regarding efi shell < worked. I am always wary of downloading random stuff of the internet but was running out of options, so pulled all the drives from my pc before trying it.

I have now flashed the bios, efi and firmware to the latest version from broadcom's web site;

LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

Adapter Selected is a LSI SAS: SAS2008(B2)
Controller Number : 0
Controller : SAS2008(B2)
<snip>
NVDATA Version (Default) : 14.01.00.05
NVDATA Version (Persistent) : 14.01.00.05
Firmware Product ID : 0x2213 (IT)
Firmware Version : 20.00.07.00
NVDATA Vendor : LSI
NVDATA Product ID : SAS9212-4i4e
BIOS Version : 07.39.02.00
UEFI BSD Version : 07.27.01.01
FCODE Version : N/A
Board Name : IBM 6Gb SAS HBA
<snip>

Finished Processing Commands Successfully.
Exiting SAS2Flash.

This appears to have fixed the issue with not being able to enter the PC's bios when it was set to ufei and the HBA was inserted.

However sas2flash still fails to see the card in Windows, while it works now in Linux. This is not a great issue but somewhat odd(?). As mentioned previously I am not using secure boot.

The other strange issue is with that ssd I had connected to the HBA. This works as expected connected directly to the mbo sata, but is ridiculously slow on the hba. Its A Crucial one and I have never had any issues with this make before now. I tried another ssd, a WD and this time and that performs as expected with reads of around 570mbs.

I've tried to attach some images showing this, but for some reason cannot. Maybe I don't have permission on this forum?

Any ideas on the strange disk behaviour? Could it be a compatibility issue? I suspect that neither of these disks would have been manufactured when the HB was made.

<edit> linked picures of disk behavior on imgur
Crutial disk - v slow

http://imgur.com/a/6qvIGuY

Western Digital SSD behaves a expected.

http://imgur.com/a/8dC71Ol

</edit>