LSI 9300-16i $129 obo

[mattlach]

This LSI HBA is my first card thats not an enterprise RAID card ever. Ive run Arecas (1280, 1280ml, 1882ix-24i) for decades with zero failures, zero data loss. Just the concept of hardware RAID itself is getting deprecated

I moved from hardware RAID to ZFS for all of my home stuff ~12 years ago now, initially through the BSD based FreeNAS, but for the last 8 years by just using ZFS on Linux, and I have been very happy with it.

Being able to recover and analyze drives on any system you can connect them to, and not being reliant on a proprietary RAID format has been a huge selling point for me. As has ZFS ability to better mitigate silent bit rot.

 

[nabsltd]

Is it really needed, or is it for some sort of special applications?

The consensus seems to be that there are some configurations that might need it, but nobody here has hit it, even running all 16 ports.

Technically, the card is only allowed to draw 25 watts from an x8 PCIe slot, and the listed max power is higher than that, but not a lot (around 35 watts, IIRC).

 

[mattlach]

The consensus seems to be that there are some configurations that might need it, but nobody here has hit it, even running all 16 ports.

Technically, the card is only allowed to draw 25 watts from an x8 PCIe slot, and the listed max power is higher than that, but not a lot (around 35 watts, IIRC).

Yeah, my bad. I didn't read the thread history. I updated my post above with what I found for reference.

Has anyone confirmed that it is just a 6pin PCIe power connector? I mean, I'll find out if it is keyed correctly when I try to plug it in, but mine hasn't arrived yet...

 

[Fritz]

None of my servers have a 6 pin PCIe connector. This is an SAS controller so it's not common for it to be used in a desktop PC. I assume newer servers have a 6 pin connector but not sure.

 

[mattlach]

None of my servers have a 6 pin PCIe connector. This is an SAS controller so it's not common for it to be used in a desktop PC. I assume newer servers have a 6 pin connector but not sure.

Yeah, my use is a little atypical being in my testbench desktop. Typical application - I presume - would be in a server. (I have my 9305-24i in my server but it doesn't have a supplemental power port)

My testbench machine is used for things like testing hard drives (badblocks, etc.) wiping hard drives, taking drive images or re-imaging drives, flashing SAS controller firmware, backups, etc. Right now it is on an old x79 i7-3930k, but when I upgrade my server I'll probably move the old supermicro server board into this system.



(man I miss when consumer hardware actually had a ton of PCIe lanes. Modern Desktop PC's are so limited.)

The old solution (9300-8i -> Intel RES2V240 Expander) randomly started dropping drives a couple of weeks ago. Not sure if it is the controller or expander, but a single card is less complex than two, so that's why I got the 9300-16i

Since it only needs ~2w above what an 8x PCIe port supplies I bet it would be pretty safe to use a 4pin auxiliary power/molex to 6pin PCIe power adapter.



I wouldn't use one of those adapters for a power hungry video card, but for 2W it should be safe. (on a power hungry video card, this could let the magic smoke out)

I feel like you can break out a spare one of those 4-pin power connectors pretty easily in most servers. At least in my Supermicro server I was able to use a splitter off the power going to the backplane for some aux power.

Honestly I'm quite surprised they just didn't put a 4pin connector like this directly on the card, as the 6pin PCIe power tends to be a consumer standard.

You know, sort of like Intel did with their SAS expanders:

 

[mattlach]

If it is indeed just a 6pin VGA/PCIe power cable they come with just about every consumer power supply (but probably not with most enterprise models) so with a little luck most of us have one kicking around that will work.

(Don't use modular power cables with power supplies they didn't come with. There is no wiring standard on the back end, so they could potentially all be different, resulting in a NASTY surprise if you mix and match)

Mine just arrived. The connector is indeed keyed exactly like a 6-pin PCIe power cable that come with consumer power supplies. I'm going to go ahead and assume this is the correct power cable, unless this is a usability blunder of epic proportions on the part of LSI

The board I am using has all 16x slots (some of them 8x electrically) so I probably don't need to plug in supplemental power, but probably will anyway since I have the power cable, and it doesn't cost me anything to be on the safe side.

Presumably some sort of additional power cable is often needed in servers these days too, considering how many Tesla style compute modules Nvidia sells for data center use. No idea what kind of power connectors those use though.

 

[nabsltd]

The board I am using has all 16x slots (some of them 8x electrically) so I probably don't need to plug in supplemental power

This doesn't actually make any difference, as the card is an x8, which means it doesn't have the extra pins needed to negotiate into "high power" mode to get the extra power.

 

[Fritz]

Went digging through my connector box today and found a dozen Molex to 6pin PCIe adapters. They're all wired the same so it must be a standard. For adapters anyway if not PSU cables.

I haven't opened the server yet to install one but probably will tomorrow. Can't hurt to have it hooked up. I hope.

 

[Fritz]

Just hooked auxiliary power to mine so I should be good to go. I used this connector -

 

[mattlach]

This doesn't actually make any difference, as the card is an x8, which means it doesn't have the extra pins needed to negotiate into "high power" mode to get the extra power.

Interesting. Did not know that it required additional pins.

I'm guessing its a good idea to always plug in the supplemental power then.

That said, maybe it is only required when using RAID firmware, as calculating checksums will put a much higher load on the card than just using it as an HBA. Still probably safer to have supplemental power connected, just in case.

 

[mattlach]

So, I got mine plugged in.

I didn't realize these cards were just two 8i's behind a PLX chip:

$ lspci |grep -i lsi
05:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 (rev 02)
07:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 (rev 02)

Also, mine came with some pretty ancient firmware, so I am going to have to update that...

$ dmesg |grep -i lsi
[    2.340995] mpt3sas_cm0: LSISAS3008: FWVersion(07.00.01.00), ChipRevision(0x02), BiosVersion(08.15.00.00)
[    2.840381] mpt3sas_cm1: LSISAS3008: FWVersion(07.00.01.00), ChipRevision(0x02), BiosVersion(08.15.00.00)

As always I'm struggling to find the right firmware on Broadcom's pathetic excuse for a webpage. It feels like they intentionally make it as obfuscated and difficult to find what you are looking for on it as possible...

I found the firmware for the 9300-16e which I presume is the same, and the only difference is that the E version has the external ports, but I am not 100% sure.

 

[mattlach]

Alright, so in my searches, the firmware for the 9300-16i does not show up anywhere on Broadcom's website.

I was able to access it by just replacing the e with an i in the download link though

I still think they should be the same, but just to be on the safe side, I'm using the file that actually says 16i...

Link to firmware here

Of course they no longer include the linux binary for flashing the firmware, only windows and DOS, so now I need to remember how to make a bootable DOS USB stick, or find an old copy of the linux command line tool... :/

Since it has two SAS3008 chips behind a PLX chip, I guess I'm just going to ahve to flash it as if it were two 9300-8i's? One at a time?

 

[Fritz]

Can it be flashed from within MSM?

 

[mattlach]

Can it be flashed from within MSM?

You know, I've been doing this for a long time, and I've never heard the term MSM before and Wikipedia's disambiguation page was not helpful

Every time I've flashed an LSI card (going back to my original IBM M1015's) it's been either via a bootable dos USB stick or using the Linux binary. Back then it was sas2flash, now it is sas3flash (but they may actually be the same util, I have no idea)

You are supposed to be able to flash it using an EFI utility as well, but I have never been able to figure that out. That and the win64 version of the app just seems to lock up when run, at least under Windows 10...

I remember struggling with making the bootable usb stick every time.

Guess I am going to have to grab Rufus again...

 

[mattlach]

Guess I am going to have to grab Rufus again...

And, that didnt work this time. Got the dreaded "failed to initialize PAL" error.

It's supposed to be due to a motherboard hardware incompatibility but I could have sworn I've successfully flashed LSI controllers from dos on this system before...

So, the Windows sas3flash app crashes instantly. No luck with the DOS one as pwer above. I can't find the native linux binary anywhere despite the fact that I could have sworn I've used it before and should have it in one of my saved files.

There are supposed to be BSD and Linux and other command line tools. They are referenced all across the documentation, but the damn downloads are nowhere to be found on the companys god awful website.

The funny thing is I remember having this problem last time I flashed an LSI SAS3 controller, and I cannot for the life of me remember how I finally solved it...

I do recall trying EFI, and failing. Did I eventually find the linux binary of sas3flash? Maybe. Cannot remember.

 

[Sean Ho]

yes, you should be able to flash it as two separate 9300 controllers (remember to give them different SAS addresses, if you wipe those). The PAL error usually means you need to use the EFI sas3flash. Try putting Tianocore EFI shell or similar on a USB key instead of using your board's builtin efi shell.

 

[mattlach]

yes, you should be able to flash it as two separate 9300 controllers (remember to give them different SAS addresses, if you wipe those). The PAL error usually means you need to use the EFI sas3flash. Try putting Tianocore EFI shell or similar on a USB key instead of using your board's builtin efi shell.

Yeah, I couldn't remember what went wrong last time, but as I am working my way through the issues I am starting to remember.

The Asus P9x79 WS motherboard won't let me in to EFI shell because of Secure Boot, but there is no BIOS option to disable secure boot. Guides online say to manually unload the Platform Key in order to disable Secure Boot, but even when I do this for the purposes of trying to get into EFI shell it still refuses, saying secure boot is on.

The whole thing is infuriating.

I eventually found a solution last time I did this, about two years ago, but I can't for the life of me remember what it was. It must have either been locating the linux binary for sas3flash, or I used another motherboard I had kicking around to flash it, but the solution has been completely purged from memory...

 

[Sean Ho]

yes, you might try popping the card into another system just to flash it

 

[mattlach]

yes, you might try popping the card into another system just to flash it

Yep,

Of all the boards I have kicking around in the office, it wasn't the one I expected that worked.

I already tested the Asus workstation board, and a Supermicro board as well. Both gave me problems.

It was this old custom build mini ITX-Small Form Factor that worked when nothing else would...



Since I don't need it, I took the opportunity to remove the bios (for faster boot times) by doing a:

SAS3FLASH -c 0 -o -e 6

to erase the first one and

SAS3FLASH -c 1 -o -e 6

to erase the second.

Then reflashed with new firmware only (no bios) with

SAS3FLASH -c 0 -o -f <firmware.bin>
SAS3FLASH -c 1 -o -f <firmware.bin>

Worked like a charm once I found a compatible board.

Side note:

These puppies get HOT even when idle in an open air situation in a cold New England winter room.

I might add an additional fan over it just to keep it reasonable.

 

[Sean Ho]

Excellent, glad you got it sorted! Kudos to you for creative problem-solving!