The Versatile SAS3008 Chipset: My Vendor Crossflashing Adventures

Sleyk

Well-Known Member
Mar 25, 2016
1,189
563
113
Stamford, CT
Refresher/Primer
(You can skip this and go further below for the meat of the stuff if you already know all of this!)

To begin, let’s talk about some definitions and applicable terms. This is not meant to be PERFECT. It is only meant to give a basic level of understanding (explained in a very simple way for those who still don’t know!)

SAS – Serial Attached SCSI – Enhanced Data transfer protocol based off the old SCSI protocol. It has many features used by the enterprise industry and usually hard drives using this data transfer protocol have more features than SATA. Also, the SAS protocol came first before the SATA protocol.

SATA – Serial ATA – Transfer protocol enhanced from the old ATA protocol. The SATA transfer protocol is derived from the SAS transfer protocol.

LSI/Avago/Broadcom – The company that produces and manufactures these SAS chipsets. LSI was the initial name. Then they were bought out by Avago and they changed their name. Then Broadcom bought out Avago, so now LSI/Avago was brought under Broadcom’s umbrella. Usually the names are now interchangeable, but most times, tech people and consumers alike still refer to the cards as LSI.

IT (IT Mode) – Initiator Target – Used to pass through all your drives to the operating system. This mode allows the operating system to control your drives. Simply put, you want IT mode for most applications besides Raid.

IR (IR Mode) – Integrated Raid – The controller prepares and controls your drives and takes care of “raid” functions. A lot of people use this for their raid function, instead of having the operating system do it. (Of note, various manufacturers and vendors have their own version of raid or IR mode. I am mostly referring to LSI’s IR mode.)

Flash/Crossflash – Using a software utility to change over or “flash” a different intended firmware onto a SAS card, making it perform in a function it wasn’t initially intended for. This is the crux of using these SAS cards, as they allow flashing/crossflashing, depending on the manufacturer.

Mbps – Megabit(s) per second (notice the lower-case letters after the first letter, M)

MB/s – Megabyte(s) per second (notice both letters are uppercase)

Gbps – Gigabit(s) per second

GB/s – Gigabyte(s) per second

1st Gen LSI SAS Chipsets – 1008 Chipset and beyond. Sample popular cards – LSI SAS 3081e, 3801e, 3442, etc).

2nd Gen LSI SAS Chipsets
– 2008 Chipset and beyond. Sample popular cards – LSI SAS 9211-8i, 8e, 9207-8i, Dell H200, H310, etc).

3rd Gen LSI SAS Chipsets – 3008 Chipset and beyond. Sample popular cards – LSI/Avago SAS 9300/9311-8i, 8e, 9302-8i, Dell HBA330, Supermicro AOC-S3008L, etc).

1.5Gbps – Also known as 1st gen SATA speed. Slow as hell. Transfer speeds top out at 187.5MB/s (Megabytes per second), but real world was very much slower at around 75MB/s transfer speeds. Nowadays, you rarely see cards BOASTING a 1.5Gbps port, but back in early 2000 and shit, you WANTED a 1.5Gbps SATA port and was proud of it too! :.)

3Gbps – Also known as 1st gen SAS (SAS I) and 2nd gen SATA (SATA II) speed. Theoretical limit is 375MB/s (Megabytes per second), but real world was about 125MB/s transfer speed. This coincided well with Hard drives, as hard drives back then transferred at roughly this speed, so hard drives weren’t really a bottleneck.

6Gbps – Also known as 2nd Gen SAS (SAS II) and 3rd gen SATA (SATA III) speed. The theoretical limit is roughly 675MB per second. Real-world is closer to about 500MB/s, with the most hard drives transferring at 150-160MB/s. This is where the age of SSD’s really took hold, as SSD’s were able to take advantage of the real-world transfer limits of 6Gbps. Most decent SSD’s will do 500-550MB/s easily.

12Gbps – Also known as 3rd Gen SAS (SAS III) speed. SATA does not have a protocol to keep up with SAS here. They did try, with the defunct 10Gbps SATA “Express” garbage, but it didn’t catch on due to various reasons. A personal dislike for me was the space and port “rent” the protocol wanted. (Please google "SATA Express" for more details.) Theoretical transfer limits for this speed are about 1.25GB/s. (Gigabytes per second, not Gigabit) Real-world is actually very close. It would seem that the SAS protocol gets better and better with every generation :.)
Real-world transfer speeds (with SSD’s in Raid 0) can easily do 1.25GB/s (one and a quarter GIGABYTES of data transferred per second).

24Gbps (Future) – The SAS people ain't messing around. They already have plans and I think, cards that are already transferring at 24Gbps. I could be wrong though. Generally, a google search will give more in-depth answers. This will be called SAS IV or SAS 4.0 if I am not mistaken. Transfer speeds will top out at an insane 2.5GB/s!!! That's two and a half GIGABYTES of data being transferred every second! This is PCI-E 4.0 level stuff right there. There are 4th Gen LSI/Avago/Broadcom SAS chipsets out already now (LSI 9500, 9561, etc).

Firmware – SAS cards need firmware in order to function. It can be IT or IR or MR (MegaRaid) firmware. Depending on your use case, you will flash and use any of these firmwares onto your SAS card.

Now that you know a little about where these terms come from, you can better understand when someone refers to first-gen sas, or 2nd-gen sas speeds and so forth, etc.

~ ~ ~ ~ ~​

Now, to the Meat.

In my testing and research, I have focused a lot on 3rd Gen LSI SAS chipset-based cards. There are abundantly plenty of resources to be found pertaining to 2nd Gen SAS cards, so I wanted to start contributing to the 3rd gen more.

I have been using 2nd Gen LSI cards for a while now. Years in fact. I recently (in the past year or so) started using 3rd Gen SAS cards. These cards use the PCI-E 3.0 slot and can transfer at almost double the theoretical limit of the SAS 2nd gen cards. Albeit, if you still utilize hard drives, there really isn’t much reason to use the 3rd gen cards, other than most computers now have 3.0 slots on board, so why not?

If using SSD’s however, you will definitely want to use a 3rd gen SAS card. If not for nothing other than a speed boost, and better compatibility with newer technology.

I have still much to learn, so don’t see me as the authority on these cards, not by a long shot, but I have been doing testing and research as much as I can, and I have helped contribute a little bit to 3rd Gen LSI crossflashing with the Dell hardware variants.


~ ~ ~ ~ ~​

My Vendor Crossflashing Adventures

So, when I first started looking for cards that use the SAS 3008 chipset, and I initially came across the famous LSI 9300/9311-8i. But there are plenty more. You have the Dell H330/HBA330, the Quanta QS 3008 Mezzanine, and of course the Supermicro AOC3008 adapter. Fujitsu also makes a 3rd gen card, called the Fujitsu CP400i or D3307. You also have the Intel 3008 adapter and mezzanine variants, and of course, the cards that helped start it all, the IBM variants.

All these cards utilize the 3rd Gen SAS 3008 chipset as the core chipset powering their cards.

As I looked at these cards, I realized that the Supermicro card (and in many instances, the LSI 9300/9311-8i) allowed Crossflash between vendors. As of right now, the Dell variants (The H330 and the HBA330) is the only card that doesn’t let you crossflash between vendors. I am doing continued testing on this, and will update if I get a breakthrough.

The Supermicro card is special, as it is usually found slightly cheaper than the LSI 9300/9311-8i, and Supermicro offers excellent support as with all their other products.

In my testing, I have found that the Supermicro card, along with many other SAS3008 chipset-based cards will Crossflash between vendors easily.

So why does this matter? How is this helpful anyway?

Well, if you use a proprietary system, like Dell or Quanta for example, you can get better compatibility by using the vendor firmware and cards for your system. Of course, other cards will work, but you get better compatibility if using the vendors own code. So, if you use for example, a Dell R730 system, you might want to use a Dell HBA330 instead of a LSI 9300-8i, although either version of card will work.

I am somewhat of a stickler of things, so if I have a Dell system, why not have your HBA card with Dell firmware? :.)

Ultimately, this is a vanity thing, and you don’t really need this, but I just wanted to outline the possibility that you CAN do it, if you want.

The hardware vendors are: (In alphabetical order, to my current knowledge)

Dell

Fujitsu

IBM

Intel

LSI

Quanta

Supermicro

So, the heart of what I am saying is that it is possible to take a specific card, from a specific vendor, and Crossflash between the vendors. This is cool right? :.)

Dell currently only uses IT firmware, with their Raid cards utilizing their own proprietary firmware roms (that’s a story for another day). IBM uses IT Firmware, Fujitsu utilizes IT firmware as well. Intel uses IT and IR firmware, LSI, being the maker of the chipset, uses IT and IR firmware of course. Quanta uses IT and IR firmware, and finally, Supermicro uses IT and IR firmware as well. Of note, all of these vendors use the same code and firmware as LSI, and just tweak it a little bit to match their products. In most cases, the firmware you crossflash is identical to the original LSI firmware.

So, you can take the Supermicro AOC-S3008L-L8i/8e card and Crossflash it to an LSI 9311-8i. But did you know that you can also turn the Fujitsu D3307 card into a Dell HBA330? Heck, wanna turn your LSI 9300-8i into a Dell HBA330 or even an IBM N2215 or Quanta card? Go ahead! Crossflash that bad boy. Ah! See, now we're getting into it!

I found that you can Crossflash nicely between vendors and make your card whatever you like. As of right now, the only exception is the manufactured Dell hardware variants. I will hopefully soon figure out how to crossflash this, but for now, Dell cards are only limited to Dell firmware. I will explain a little further down.

Here’s what I found:

Supermicro AOC-S3008L-L8i/L8e can inter-vendor Crossflash to:

A LSI 9300/9311-8i

A Dell HBA330

A Fujitsu CP400i/D3307

A Quanta QS 3008

A Intel RS3U080

A IBM M1215/N2215
Fujitsu D3307/CP400i can inter-vendor Crossflash to:

A Supermicro AOC-S3008L-L8i/L8e

A Dell HBA330

A LSI 9300/9311-8i

A Quanta QS 3008

A Intel RS3U080

A IBM M1215/N2215​

LSI 9300/9311-8i can inter-vendor Crossflash to:

A Supermicro AOC-S3008L-L8i/L8e

A Dell HBA330

A Fujitsu CP400i/D3307

A Quanta QS 3008

A Intel RS3U080

A IBM M1215/N2215​

Quanta QS 3008 Mezzanine can inter-vendor Crossflash to:

A LSI 9300/9311-8i

A Dell HBA330

A Fujitsu CP400i/D3307

A Supermicro AOC-S3008L-L8i/L8e

A Intel RS3U080

A IBM M1215/N2215​

Intel RS3U080 (if you have one) can inter-vendor Crossflash to:

A LSI 9300/9311-8i

A Dell HBA330

A Fujitsu CP400i/D3307

A Supermicro AOC-S3008L-L8i/L8e

A Quanta QS 3008

A IBM M1215/N2215
IBM M1215/N2215 can inter-vendor Crossflash to:

A LSI 9300/9311-8i

A Dell HBA330

A Fujitsu CP400i/D3307

A Supermicro AOC-S3008L-L8i/L8e

A Quanta QS 3008

A Intel RS3U080​

Dell HBA330/H330 can inter-vendor Crossflash to: Nothing else as of yet. This card has some Dell protection on it, possibly embedded in the manufacturer pages, and is currently not allowing an inter-vendor Crossflash. This is true of a crossflashed H330 and a manufactured true HBA330. Both cards will not currently allow a different vendor Crossflash. This might be due to Dell’s manufacture page 2 protection. I am working on trying to bypass this, but it may call for either a linux low-level hostboot, or perhaps a modified sas3flsh to bypass this security. We’ll see. Since the Dell firmware is in fact, a tweaked, slightly modified version of LSI’s IT firmware, the Dell firmware can be flashed over to any other card successfully. This is pretty useful if you have a proprietary Dell system, and you have a LSI or Supermicro SAS3008 card. You can just make it a Dell HBA330, so that’s good, but I’m pissed that Dell wanted to prevent vendor crossflash for its own product. What a crock. What a piece of work Dell is. Always gotta be that one difficult vendor. I love them, but I hate ‘em too. :.)

Small note on Mezzanine Cards

Of note, I currently do not own a Quanta or Intel Mezzanine card or proprietary system. I derived that it is possible to crossflash these mezzanine cards, as they utilize the LSI firmware specifically, with just a product ID name change in the firmware, with no manufacturer page 2 protection. I also came to this conclusion, as the Supermicro and LSI cards will accept and take the Quanta product ID firmware with zero problems. Hence, crossflashing back and forth between vendors with these 2 manufacturer mezzanine cards is a strong and high possibility.

~ ~ ~ ~ ~​

So there you have it! Crossflash your 3rd Gen SAS 3008 cards to whatever vendor you like. If you have a Dell hardware variant, it is the only card that won’t crossflash for now, but I suspect we’ll figure it out soon. For now, go grab that Supermicro AOC-S3008L-L8i/8e or that LSI 9311-8i and crossflash that bad boy to your hearts content!

Remember, to crossflash between vendors, its better to do a "-o -e 7", than an "-o -e -6" erase in sas3flsh/sas3flash to erase the MPB pages. You may run into issues if you use -o -e 6. Just save or write down your sas address. That's all you need. You can always convert back to your original vendor anyway.

Google Drive with All Latest Vendor Firmware (sas3flsh and sas3flash.efi included): SAS 3008 Firmwares.zip

~ ~ ~ ~ ~​

Some Reading Links for all Hardware Vendors and their Cards

Supermicro AOC-S3008L-L8i/8e - AOC-S3008L-L8e / AOC-S3008L-L8e+ | Add-on Cards | Accessories | Products - Super Micro Computer, Inc. and/or AOC-S3008L-L8i | Add-on Cards | Accessories | Products - Super Micro Computer, Inc.

Quanta QS 3008 Mezzanine Card - QS 3008 | QCT SAS Mezzanine | QCT.io

Fujitsu D3307/CP400i - PRAID CP400i - 8 port 12Gb/s SAS, 6Gb/s SATA - Fujitsu CEMEA&I

Intel RS3UC080J/RS3UC080/RS3WC080 - Intel® RAID Controller RS3UC080J or Intel® RAID Controller RS3WC080 or Intel® RAID Controller RS3UC080

IBM N2215/M1215 - N2215 12Gb SAS/SATA Internal HBA Product Guide > Lenovo Press and ServeRAID M1215 SAS/SATA Controller Product Guide > Lenovo Press

Dell HBA330 – (Click on the link to go to the “Series 9” adapter set of cards) List of PowerEdge RAID Controller (PERC) types for Dell EMC systems | Dell Vietnam

LSI/Avago/Broadcom 9311-8i/9300-8i - SAS 9311-8i Host Bus Adapter and/or SAS 9300-8i Host Bus Adapter
 
Last edited:

nthu9280

Well-Known Member
Feb 3, 2016
1,625
461
83
San Antonio, TX
Great info. Can you add IBM / Lenovo to the vendor list. The initial cross flash started with them and the guide is on this forum.
M1215 - 9340-8i -> 9300-80 IT
N2215 - 9302-8i (If you recall, there was a firestorm over my great deals thread :eek: )

Both of these can be flashed to 9300-8i (IT)

I addition, Intel on 2308-8i (RMS25KB080) cards used to have a lock to work only on their server motherboards even after flashing to IT. Not sure if they continued similar approach to SAS3008 and after.

I've read here on the forums regarding the vendor design modifications to the Broadcom's reference design so that could be a reason for this as well as your issues with the Dell H330/HBA330
 
  • Like
Reactions: Sleyk

IamSpartacus

Well-Known Member
Mar 14, 2016
2,350
572
113
Has anyone been able to successfully crossflash an LSI 9300-8e to IR mode. I can't seem to get it to go. I get the following:

 
  • Like
Reactions: Sleyk

Sleyk

Well-Known Member
Mar 25, 2016
1,189
563
113
Stamford, CT
Great info. Can you add IBM / Lenovo to the vendor list. The initial cross flash started with them and the guide is on this forum.
M1215 - 9340-8i -> 9300-80 IT
N2215 - 9302-8i (If you recall, there was a firestorm over my great deals thread :eek: )

Both of these can be flashed to 9300-8i (IT)

I addition, Intel on 2308-8i (RMS25KB080) cards used to have a lock to work only on their server motherboards even after flashing to IT. Not sure if they continued similar approach to SAS3008 and after.

I've read here on the forums regarding the vendor design modifications to the Broadcom's reference design so that could be a reason for this as well as your issues with the Dell H330/HBA330
Haha! Yes sir! First post updated! Credit and Thanks to Nthu! :.)
 
Last edited:

Sleyk

Well-Known Member
Mar 25, 2016
1,189
563
113
Stamford, CT
Has anyone been able to successfully crossflash an LSI 9300-8e to IR mode. I can't seem to get it to go. I get the following:

Hmm, I havent played around with a 9300-8e, but the procedure should be the same I think. That is somewhat strange to see that the 9800e needs nvsram. I thought this was associated with a battery?

You could try the Quanta IR firmware, which is just LSI IR firmware anyway. Then probably inter-vendor flash back to LSI? Worth a try
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,350
572
113
Hmm, I havent played around with a 9300-8e, but the procedure should be the same I think. That is somewhat strange to see that the 9800e needs nvsram. I thought this was associated with a battery?

You could try the Quanta IR firmware, which is just LSI IR firmware anyway. Then probably inter-vendor flash back to LSI? Worth a try
Just tried to download the Quanta IR firmware for the 8e and it says file not available.
 

Sleyk

Well-Known Member
Mar 25, 2016
1,189
563
113
Stamford, CT
No dice, same error. I guess the 8e cannot be flashed to IR mode. Bummer.
Sorry to hear that brother! I was hoping it would work. Now we need to investigate this nvsram thing to give us a better idea of why it wont work. I wonder if its an extra "block" of memory that the chip is supposed to have, and since it claims isn't present, it doesn't allow a flash.
 

VJO

New Member
Jun 6, 2020
2
0
1
Hello everyone, thanks for the nice information, this is going to be my first build, so here goes my doubt, planning to get "8 Port 12Gbps LSI SAS 3008 Quanta 3008-8i-IR QS 3008 Storage Mezzanine Card", as a HBA, to be used in unraid. To be installed in supermicro motherboard...will it work...(why am I choosing that particular card? Becoz it's cheap, when compared to other sas3 hba's.) Don't know whether will it work..., if yes my next question is how do I connect it to my pcie slot...any special adapter required... kindly advise...
 
Last edited:

pricklypunter

Well-Known Member
Nov 10, 2015
1,663
487
83
Canada
Hello everyone, thanks for the nice information, this is going to be my first build, so here goes my doubt, planning to get "8 Port 12Gbps LSI SAS 3008 Quanta 3008-8i-IR QS 3008 Storage Mezzanine Card", as a HBA, to be used in unraid. To be installed in supermicro motherboard...will it work...(why am I choosing that particular card? Becoz it's cheap, when compared to other sas3 hba's.) Don't know whether will it work..., if yes my next question is how do I connect it to my pcie slot...any special adapter required... kindly advise...
No, that card requires a matching Mezzanine socket on the Mainboard. I believe you will be wanting a PCIe version for the SM Mainboard :)
 

ironman

New Member
Jan 11, 2021
2
2
3
@Sleyk

Did you ever crossflash a Fujitsu CP400i/D3307 yourself? And if so, could you please further elaborate on how you did this exactly?

The only source I've been able to find on how to crossflash a CP400i/D3307, is this video tutorial:


However, it requires a valid SBR of a target LSI 9300-8i to not brick the controller during the process. But not having a 9300-8i is the reason why I'd like to crossflash a CP400i in the first place. SBR requests in the Youtube comment section went unheard, and I have not been able to find it anywhere else on the internet either. I did manage to gather all the other files needed in that tutorial, so if you - or anyone else reading this - could provide me a vaild .sbr dump of a working LSI 9300-8i I'd be very happy!

Also, can you tell me if it's possible to intra-vendor flash a Fujitsu PRAID CP400i/D3307 to a PSAS CP403i/D3327, perhaps even more easily? Both controllers share the same hardware and only differ in firmware according to official Fujitsu information:
Name:
PSAS CP403i

Description:
- SAS/SATA Host Bus Adapter based on LSI MegaRAID SAS3008
- for internal SAS/SATA HDD and SSD
- No Cache
- controller HW identical to PSAS CP400i
- IT FW- Stack without RAID functionality, other than PSAS CP400i
Sources:
CP403i (S26361-F3842-E3) Information
PRIMERGY RX2540 M5 Manual rsp.

Thanks!
 
Last edited:
  • Like
Reactions: Sleyk