Info on LSI SAS3408? Got myself a 530-8i on eBay...

[nthu9280]

StorCLI is summarized in the previous posts in this thread. You can always update to the latest Lenovo FW with Windows / Linux. If you want turn it into HBA, you need a mobo with uefi support. Not sure if you can do that in other OS but I've not had much luck.

 

[Gadgetguru]

Yeah, I see all the StorCLI commands, just don't know how to get into/use StorCLI... If I did I could follow iblik.94 instructions but I don't know how to get to that point. I'll keep trying and searching as time allows... Thanks.

 

[nthu9280]

Broadcom Download site:

Support Documents and Downloads

You will need to download the driver for your OS, FW and StorCLI (multi-os package).
You may also need the 9440-8i driver too since the card comes with IR FW.

 

[Gadgetguru]

OK, 7 hours later...

What a pain in the...

Used LSI Storage Authority to try and update the firmware. Said the firmware file was corrupt.

Was able to figure out using StorCLI, or so I thought, but nothing would work, I kept getting errors.

So my next idea was to remove the card from my server and put it in my main computer since it has UEFI. Did that, used shell.efi on a thumb drive, had a few issues but eventually got into the shell. Sweet! Started following the instructions iblik.94 gave but I was getting errors again. More research was required.

First problem: storcli / cx download firmware file = HBA_9405W-16i_SAS_SATA_Profile.bin

So apparently the x in cx is supposed to be a number. The adapter number. Since I only have one, the number was zero.

Second problem: Where is says 'download firmware file' I didn't realize that was supposed to be typed out/included. So I was only typing storcli /c0 HBA_9400-8i_SAS_SATA_Profile.bin

It still wouldn't work typing it all out. I looked through the storcli commands and decide to try only 'download file'. And bam, it worked!

Followed the rest of the instructions, put it back into my server, and booted it up. Everything looked good when booting, and then I went back into LSI Storage Authority. It now says HBA 9400-8i, everything is updated, and all my drives are visible.

I did say I needed it Barney style! LOL Took me all day but got it figured out.

 

[james23]

It seems most of you all are trying to flash this 530-8i over to IT mode, but if i wanted to use this to RAID 1 , 2x nvme drives, would that work? (maybe with the lenovo FW that it comes with, although lenovo sheet does not mention tri-mode nor nvme..) - or would it be possible to use the 9440-8i (raid) FW instead of the 9400-8i (hba/IT) FW you all have been using with success?

-- ideally to my white tipped supermicro sff-8643 cables that go from my SM 4 port nvme card to my supermicro nvme backplane (its 12x sas3 backplane, 4x of which are nvme).

(im not looking for IT mode, and im hoping that the "tri-mode" would allow me to connect nvme's).
thanks

EDIT: interesting thread, but in chinese (use google translate):
(looks like you do need that 75-100$ cable, but i can only find it in u.2 so im not sure how i would connect this to a backplane. looks like you can only connect it directly to the u.2 on the back of a 2.5" nvme

关于Broadcom(LSI) 9400 Series NVMe线材的可选方案(更新可用线材) - 第2页 - 电脑讨论 - Chiphell - 分享与交流用户体验

EDIT2: OK HERE IS most likely the answer. you need a 94xx , and then one of these EXPENSIVE cables (75-125$) to connect to your nvme's . i may try this but would rather buy a 70$ 530-8i and flash to 9440 than buy a ~200$ 94xx-8i from ebay (if former is possible)..

 

[andrewbedia]

Can confirm that you cannot use the cheap intel cables. I don't know if you have to use the LSI cables, but for sure the intel ones do not work. I have one of the expensive LSI 8643 to direct connect cables (2nd to last in that picture) and it works, but getting terrible performance. See https://forums.servethehome.com/ind...ance-on-broadcom-9440-8i-lenovo-530-8i.23534/

I really wish Patrick or someone with a lot of time and different pieces of hardware could do a long review of this card in the context of being used with NVMe.

 

[james23]

I really wish Patrick or someone with a lot of time and different pieces of hardware could do a long review of this card in the context of being used with NVMe.

THIS-- there are ZERO reviews for this line of lsi cards. i wonder if its bc they all know the nvme performance is horrible? (im not saying its horrible, i dont know, its just very werid that there is not ONE review of this somewhat important, unique, HW in terms of hot nvme tech (ie nvme+raid), 2+ years since the 94xx line has been out! ).

edit; also THANK you this info (im not sure how i didnt find your thread in all my searching)- you may have saved me ~ 1000$

 

[nezach]

Here are a bit more detailed flashing instructions for people brand new to this + some additional information.

Avago Tri-Mode cards support UEFI HII (Human Interface Infrastructure) configuration utility. What that means is that you might not have an option to hit CTRL+... during boot process and enter configuration screen like with older cards. The configuration options will instead appear somewhere inside your regular UEFI configuration screen. If and where exactly it appears will depend on your UEFI. This is how it looks in MSI's UEFI.

If you want to use HII config utility and you are using Secure Boot you need to flash signed rom file. If you flash regular file config utility will not appear. If you don't use Secure Boot then it does not matter if you flash signed or unsigned file.

These cards also no longer have IR firmware as such.
9400-8i firmware archive has two files HBA_9400-8i_SAS_SATA_Profile.bin and HBA_9400-8i_Mixed_Profile.bin. I am assuming SAS_SATA allows you to use SAS/SATA drives and Mixed_Profile allows SAS/SATA/NVMe drives, but I am not 100% sure about this.
9440-8i firmware archive has a single MegaRAID file 9440-8i_nopad.rom

Now onto flashing:

• Download required files:
  • Go to HBA 9400-8i Tri-Mode Storage Adapter
  • Click on 'Firmware' and get latest fimware file: 9400_8i_Pkg_P10_SAS_SATA_NVMe_FW_BIOS_UEFI.zip
  • Click on 'Management Software and Tools' and download latest STORCLI utility: STORCLI_SAS3.5_P10.zip
• Create a backup. You can do this in EFI shell or Windows/Linux. In Linux you need to run these commands as root, in Windows as administrator.
  • #storcli show all
    lists all available adapters; note the controller index, you will need to use it in all subsequent commands (/cx - specifies the controller where X is the controller index)
  • #storcli /c0 show all
    dumps all information about the controller. I recommend saving the whole output, at a minimum write down the SAS Address
  • Backup misc flash regions to files. Just in case you need them later.
    #storcli /c0 get bios file=backup_bios
    #storcli /c0 get firmware file=backup_firmware
    #storcli /c0 get mpb file=backup_mpb
    #storcli /c0 get fwbackup file=backup_fwbackup
    #storcli /c0 get nvdata file=backup_nvdata
    #storcli /c0 get flash file=backup_flash

• If you are flashing from MegaRAID firmware to IT firmware:
  • Copy below files to USB drive:
    • ..\EFI\storcli.efi
    • ..\Firmware\HBA_9400-8i_SAS_SATA_Profile.bin
    • ..\SAS35BIOS_Rel\mpt35sas_legacy.rom (optional)
    • ..\UEFI_BSD_HII_SAS3.5_IT_X64\mpt35sas_x64.rom (optional) if secure boot is not used
      or
      ..\UEFI_BSD_HII_SAS3.5_IT_X64\Signed\mpt35sas_x64.rom (optional) if secure boot is used
  • Place jumper on J4 SBR (Serial Boot Rom) header.
  • Boot into EFI shell.
  • Flash firmware:
    #storcli /c0 download file=HBA_9400-8i_SAS_SATA_Profile.bin
    once complete it will show: Description = CRITICAL! Flash successful. Please power cycle the system for the changes to take effect
  • Power off.
  • Remove jumper from J4 header.
  • Boot into EFI shell or OS.
  • Flash legacy BIOS. This step is optional, if you are not sure what this is do not flash it. There is no easy way to remove this, but this can be added later.
    #storcli /c0 download bios file=mpt35sas_legacy.rom
    
  • Flash UEFI BSD - UEFI HII config utility. This step is also optional. There is no easy way to remove this, but this can be added later. Again, flash signed/unsigned file depending if you use Secure Boot.
    #storcli /c0 download efibios file=mpt35sas_x64.rom
  • Set SAS address
    #storcli /c0 set sasadd=500...
  • Set assembly number
    #storcli /c0 set assemblynumber=03-5000...
  • Set tracer number/serial number
    #storcli /c0 set tracernumber=SP...

• If you are flashing from IT firmware to IT firmware. Upgrading or switching from SAS_SATA to Mixed profile. Run this from EFI shell or OS (no need to set J4 jumper):
  • #storcli /c0 download file=HBA_9400-8i_SAS_SATA_Profile.bin

 

[nezach]

Has anyone been able to flash 9440-8i's MegaRAID firmware after flashing to IT firmware or straight from Lenovo's MR firmware? I tried a bunch of things, but nothing seems to work. Storcli keeps saying "The image checksum is invalid." when i try to do it.


P10 storcli archive contains not just storcli utility but also storclio. I went googling to find out what it is and SuperMicro FAQ has this interesting answer:

Due to LSI agreement, we are not allowed to release this Storclio utility software. So we cannot tell the different between the two.

It must be some kind of OEM only version of storcli that shouldn't have been publicly released. I did find at least one command that storclio supports, but storcli doesn't:

storclio /cx set SubOEMId=<val>
DESCRIPTION: This command sets Sub OEM ID field of the MFC features to <val>

Maybe this utility will come in handy.

 

[james23]

These cards also no longer have IR firmware as such.

crap, what is this line about? i was hopeing to try to flash my 530-8i to IR FW and see how 2x RD1 nvme's performed.

Does this mean that even a native lsi 9440-8i can *only* be used as an hba / it fw?

Ill find out some of these answers in about 2 days when my 530-8i arrives and i try this, but if you have any info on that line above, please provide. (also thanks for that awesome, clear, post/how to guide. very helpful!)

 

[nthu9280]

@nezach thanks for the nice write-up!

Couple of adds -
- AFAIK, 9440-8i is IR version for these cards. Unlike the previous SAS2008, SAS2308 based cards.

- I noticed that the SAS address, Tracer#, Assembly# and Board Name are wiped when flashed to 9400-8i with J4 jumper on. Did yours stay? No big deal but they are on stickers as well as in the first page of "StorCLI /c<x> show all >> adpa<x>.txt" where x is controller #.
Please add to your write-up so it is comprehensive.

Flashing back to 9440-8i
Have you tried with jumper J4 on? IIRC, there are -noversionchk and -nosigchk options. I'll give a try this weekend.

Looks like they are releasing FW updates on almost monthly. I think it was P8 when I did few weeks ago.




-

 

[nezach]

crap, what is this line about? i was hopeing to try to flash my 530-8i to IR FW and see how 2x RD1 nvme's performed.

Does this mean that even a native lsi 9440-8i can *only* be used as an hba / it fw?

Ill find out some of these answers in about 2 days when my 530-8i arrives and i try this, but if you have any info on that line above, please provide. (also thanks for that awesome, clear, post/how to guide. very helpful!)

Well, there are 3 types of firmware IT, IR and MR (MegaRAID). See my recent post about this: https://forums.servethehome.com/ind...isting-plus-oem-models.599/page-8#post-220148

IR and MR can both do RAID. 530-8i is MegaRAID card so it already comes with MR firmware, just like 9440-8i, but this card uses Lenovo version of firmware. I don't know what the differences are between Lenovo's MR and Avago's MR firmware.

As far as flashing 9440-8i's MR firmware to 530-8i, you will have to experiment with it. Just don't flash it to IT mode, cause we don't yet know if it is possible to go back to MR firmware.

 

[nezach]

- AFAIK, 9440-8i is IR version for these cards. Unlike the previous SAS2008, SAS2308 based cards.

Hmm, interesting. Don't want to muddy things even more but sometimes Avago mentions iMR (integrated MegaRAID), I don't know if that is same as IR or not. Maybe iMR is MR running on controllers with IOC chips instead of RoC.

- I noticed that the SAS address, Tracer#, Assembly# and Board Name are wiped when flashed to 9400-8i with J4 jumper on. Did yours stay? No big deal but they are on stickers as well as in the first page of "StorCLI /c<x> show all >> adpa<x>.txt" where x is controller #.
Please add to your write-up so it is comprehensive.

Mine are gone along with the serial number. Before I even flashed my card to IT firmware I "accidentally" ran Lenovo's firmware update tool lsiStorUpdate and it wiped my card clean without flashing anything to it. It must have executed "storcli erase all" or something like it. Anyway, I was not sure if serial and all other data being gone was because of lsiStorUpdate fu**-up or result of flashing IT firmware.

I wonder where all that data is located. Possibly manufacturing region/mpb or vpd. Command "storcli /cx get vpd file=<fileName>" does not work with IT firmware and I did not have a chance to try it with MR. Maybe someone with virgin 530-8i can backup mpb/vpd regions and post them here.

Flashing back to 9440-8i
Have you tried with jumper J4 on? IIRC, there are -noversionchk and -nosigchk options. I'll give a try this weekend.

I tried a lot of things J4 on/off, flashing from EFI and Linux, nosigchk and force options, storcli and storclio commands, Avago and Lenovo firmware files. Result is always the same. I might have to try flashing with Lenovo's lsiStorUpdate again.

BTW these cards seem to be resilient to bricking. Even if you erase everything and OS does not see the card, you can boot into EFI shell and flash it.

 

[james23]

Well, there are 3 types of firmware IT, IR and MR (MegaRAID)..... I don't know what the differences are between Lenovo's MR and Avago's MR firmware.

As far as flashing 9440-8i's MR firmware to 530-8i, you will have to experiment with it. Just don't flash it to IT mode, cause we don't yet know if it is possible to go back to MR firmware.

Thanks nezach, (and again thanks for that write up/guide!). Im pretty familiar with LSI cards and their different FW types, thus my question/concern on your line "These cards also no longer have IR firmware as such." (as im hoping to try nvme raid1, thus need IR not IT fw) Eitherway, i will be testing 530-8i flash to IR in about 1 day and will update here. (and hopefully get some benchmarks on nvme rd1, however i have not bought that expensive LSI nvme cable just yet, im waiting to see how IR flash goes)

This may also clear up some confusion with my question above: I have read that the 530-8i 's stock lenovo FW (raid fw, out of box), does NOT support tri-mode / nvme's - only SATA/SAS. Thus this is why i need to flash my 530-8i to LSI iR FW (to get the tri-mode / nvme support).
tks

 

[nezach]

@james23, when you get your card can you please backup mpd/vpd regions and post your files. FYI, they might contain serial number and sas address.

I looked up Lenovo cards in VMware HCL and there are two of them:

Model : 7Y37A01082 - ThinkSystem RAID 530-8i PCIe 12Gb Adapter
Device Type : SAS/SATA-RAID
VID : 1000
DID : 0017
SVID : 1d49
SSID : 0500

Model : ThinkSystem RAID 530-8i Dense Adapter
Device Type : SAS-SATA-NVME Tri-mode RAID
VID : 1000
DID : 0017
SVID : 1d49
SSID : 0502

So you might be right about these cards not supporting NVMe. Lenovo firmware package has different firmware files for these cards. There is also a file 'ctlr-info.txt' with SSIDs and corresponding firmware files.

0x502,ThinkSystem RAID 530-8i Dense Adapter,9440-8i-d_nopad.rom,
0x500,ThinkSystem RAID 530-8i PCIe 12Gb Adapter,9440-8i_nopad.rom,
0x501,ThinkSystem RAID 530-4i Flex Adapter,9440-4i_nopad.rom,

 

[james23]

thanks, will do on posting my backed up files here. FWIW- what you say above from vmware HCL, matches exactly what i read in a lenovo PDF (still can find PDF though).

the ThinkSystem RAID 530-8i Dense Adapter is a mezzanine card for lenovo servers only. (and the pdf showed this being tri-mode, but the PCIe version of their 530-8i as being sas/sata ONLY).

thus im hoping flashing to lsi 9440 ir will get me tri-mode on the pcie version (wishful thinking maybe). will update this weekend

EDIT; here is a link to a PDF of the various cables needed on the 94xx cards FOR NVME (see last page of this pdf, page10)
Storage Solutions Selection Guide (457 KB)

 

[nthu9280]

@nezach - PMed you

 

[andrewbedia]

These add in cards support NVMe, but not out of the box. You need to flash broadcom mixed-mode firmware on it then NVMe will work. It's dog ass slow though... https://forums.servethehome.com/ind...ance-on-broadcom-9440-8i-lenovo-530-8i.23534/

 

[nezach]

@nthu9280 Where do you see Tracer#, Assembly# and Board Name? I do not see them in the "show all" output. I tried programming tracer# using

./storcli /c0 set tracernumber=1234567890

and it worked fine.

Tried flashing MR firmware using Lenovo tools, but that did not work. It just complains that there are no supported cards. I think Lenovo's lsiStorUpdate tool looks for VPD and since "get vpd" command does not work with IT firmware it might not be possible to use it.


It is possible to program VPD using

storcli /c0 set updatevpd file=vpd.txt

but file needs to be a text file in the following format

;Large Resource Type ID String Tag. Describe the data type.
LR_ID_STRING_TAG = 0x82
;Length of the ID String. Must match the length of the LR_ID_STRING.
LR_ID_STRING_LENGTH = 0x001F
;Data. The actual ID String to describe the device this VPD data represents.
LR_ID_STRING = "PCIe2 SAS Adapter Dual-port 6Gb"
;Large Resource Type VPD Tag.
LR_VPD_TAG = 0x90
;Total Length of all the VPD keyword, length and data wrapped by a Large
;Resource Type VPD Tag.
LR_VPD_LENGTH = 0x004C
;VPD Keyword. Part Number of Assembly.
VPD_KEYWORD = "PN"
VPD_LENGTH = 0x07
VPD_DATA = "45D8028"
;VPD Keyword. Engineering Data Code.
VPD_KEYWORD = "EC"
VPD_LENGTH = 0x07
VPD_DATA = " G37068"
;VPD Keyword. CC - Field Service CCIN.
VPD_KEYWORD = "CC"
VPD_LENGTH = 0x04
VPD_DATA = "763A"
;VPD Keyword. Reserved and Checksum.
VPD_KEYWORD = "RV"
VPD_LENGTH = 0x01
VPD_DATA = "CHECKSUM"
;Small Resource Type End Tag.
SR_END_TAG = 0x79

once programmed you retrieve it using

# ./storcli /c0 show vpd
CLI Version = 007.0912.0000.0000 Dec 27, 2018
Operating system = Linux 4.20.3-arch1-1-ARCH
Controller = 0
Status = Success
Description = None


-----------------------------------------------------------
                      VPD Information
-----------------------------------------------------------
 Product Name   : PCIe2 SAS Adapter Dual-port 6Gb
 PN             : 45D8028
 EC             :  G37068
 CC             : 763A
 RV             : 0x5b
 x             :
              :
              :
              :
              :
              :
              :
              :
              :
              :
              :
              :
              :
              :
              :
-----------------------------------------------------------

Avago SAS3 Management Utility: VPD information incomplete.

But command "storcli /cx get vpd file=<fileName>" is not supported with IT firmware. Maybe VPD works differently in MR and IT firmware.

 

[nezach]

I compared verbose help of storecli vs storeclio and storeclio supports some interesting commands:

NAME: Set Sasaddress
SYNTAX: storclio /cx set sasaddress=xxxxxxxx
DESCRIPTION: Command Sets 64B length SAS Address to the controller.
CONVENTION:
/cx - specifies the controller where X is the controller index


NAME: Get Factory settings
SYNTAX: storclio /cx get factorysettings file=<filepath>
DESCRIPTION: Get/write all Factory settings to a given file
CONVENTION:
/cx - specifies the controller where X is the controller index


NAME: Set Factory settings
SYNTAX: storclio /cx set factorysettings file=<filepath>
DESCRIPTION: set/read all Factory settings from a given ini file
CONVENTION:
/cx - specifies the controller where X is the controller index


NAME : SET VPD

SYNTAX:
storclio /cx set vpd file=<fileName>

DESCRIPTION: Set the VPD data. Give the vpd file as input.
CONVENTION:
/cx - specifies the controller where X is the controller index


NAME: Set OOB mode on controller

SYNTAX: storclio /cx set oob mode=i2c|pcie maxpayloadsize=xxxx maxpacketsize=xxxx

DESCRIPTION:Set out of band management communication mode
    with the transport mode parameters
mode           : Mode of transport. Can be i2c or pcie
maxpayloadsize : Maximum payload size to be set for OOB communication.
maxpacketsize  : Maximum packet size to be set for OOB communication.
CONVENTION:
/cx - specifies the controller where X is the controller index


NAME: Show OOB mode on controller

SYNTAX: storclio /cx show oob

DESCRIPTION:Shows Out Of Band management communication mode
            with the transport mode parameters
mode           : Mode of transport. Can be i2c or pcie
maxpayloadsize : Maximum payload size to be set for OOB communication.
maxpacketsize  : Maximum packet size to be set for OOB communication.
CONVENTION:
/cx - specifies the controller where X is the controller index

Note: This command supports both IT and MR Controllers


NAME: Set pcilinkspeed
SYNTAX: storclio cx set  pcilinkspeed = <speed>
DESCRIPTION: sets pcilinkspeed
CONVENTION:
<speed> can be 1,2 or 3 where 1-(2.5GT/s), 2-(5GT/s, 3-(8GT/s)
/cx - specifies the controller where X is the controller index


NAME: Show SubOEMId of the controller

SYNTAX: storclio /cx show SubOEMId

DESCRIPTION: Displays the SubOEMId of the controller

CONVENTION:
    /cx - specifies the controller where X is the controller index


NAME: Set SubOEMId of the controller

SYNTAX: storclio /cx set SubOEMId=<val>

DESCRIPTION: This command sets Sub OEM ID field of the MFC features to <val>.

CONVENTION:
/cx - specifies the controller where X is the controller index



NAME: Set MFG data of the controller

SYNTAX: /cx set MFG file=<fileName>

DESCRIPTION: This command sets MFG data from the given file.

CONVENTION:
/cx - specifies the controller where X is the controller index


NAME: Clears NVRAM on Adapter.

SYNTAX: storclio /cx delete NVRAM

DESCRIPTION: This command Clears the NVRAM on Adapter.
CONVENTION:
/cx - specifies the controller where X is the controller index


NAME: Get Hardware Device page

SYNTAX: storclio /cx get HWDevice deviceType=<val>[bus=<val> slaveAddress=<val>]
[[offset=<val>] size=<val>] file=<filepath> [fileone=<filepath> filetwo=<filepath>]

DESCRIPTION: This command gets the hardware device information and fills
the user specified file.

OPTIONS:
bus: Bus from which data to be read
slaveAddress: slave address of the device in Hex
offset: Starting Offset value in Hex.
size : Total size in bytes.If the size is zero,then all data on the device
 will be got.
device type. The values are:
        TMM-EEPROM - 3
        SBR - 4
        Generic EEPROM device - 5
fileone:This optional file option is used to specify the hostboot image when the firmware is in non-operational state.
filetwo:This optional file option is used to specify the filename to which the data should be written
when the firmware is in non-operational state.
CONVENTION:
/cx - specifies a controller where X is the controller index.


NAME: Set Hardware Device page

SYNTAX: storclio /cx set HWDevice deviceType=<val> [bus=<val> slaveAddress=<val>]
[offset=<val>] file=<filepath> [fileone=<filepath> filetwo=<filepath>]

DESCRIPTION: This command sends the user specified file to firmware to
modify the hardware device information.

OPTIONS:
bus: Bus from which data to be read
slaveAddress: slave address of the device in Hex
offset: Starting Offset value in Hex.
device type. The values are:
        TMM-EEPROM - 3
        SBR - 4
        Generic EEPROM device - 5
fileone:This optional file option is used to specify the hostboot image when the firmware is in non-operational state.
filetwo:This optional file option is used to specify the filename from which the data should be read and written to the respective devicepage
when the firmware is in non-operational state.
CONVENTION:
/cx - specifies a controller where X is the controller index.

Most of them don't work against IT firmware though.