Broadcom 9500-8i, NVME U.2/U.3, Tri-Mode
- Thread starter john389
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I had a question for any of you owners of a 9400 or 9500 HBA that is unrelated to the main topic. There are multiple reports out there of people using older HBA chipsets such as 2208/2308/3008 where the HBA prevents the processor from going into deeper idle states than C2 or C3. I have not seen anything from anyone with a more modern chipset such as the 3408 or 3808 chipset. Broadcom advertises more power saving features starting with the 3408 such as " – Slumber and partial power mode support for SAS/SATA devices ", " Programmable SAS link power down", and " – Variable PCIe bandwidth negotiation". I was curious if any of you have investigated what idle states your system can achieve with powertop or similar program?
@Stankyjawnz: I can perhaps - can't promise it though - test that for you in the coming weeks, no idea when yet. However, not being able to achieve higher C-States, i.e. > C3, typically happens whenever you have high performance I/O chips in your system, be they for data storage or networking. You just introduce latency into your system, when the CPU continually enters deeper sleep states, which take more time to wake from again. The question is then how much additional energy saving you could in theory achieve by enabling deeper C states relative to problems introduced in your system. Most HBAs and networking equipment, such as SFP+ and SFP28 cards, where I know the problem from as well, are all built for enterprise environments, where latency problems are to be avoided. So you probably won't have any luck there trying to force something that is not designed for it.
However, as you can see from the datasheets - I haven't measured it yet - the newer 9400 and especially 9500 series HBAs use a lot less power than previous generations, so if you can upgrade your card, you'll perhaps save more power than what you would get from deeper C states - which you then also get if it is so designed.
9500-8i, 6W typical
9400-8i, 10W typical
9300-8i, 13W typical
Of course the max value is quite a bit higher. So newer chips are more efficient and I've stopped buying the older chipsets a few years ago.
EDIT:
The newer 9500 series chips, i.e. SAS3808 and SAS3816, are also found in, for example, Lenovo and Dell HBA cards.
I think it is the Dell HBA350i, Dell HBA355i, Lenovo 440-8i and Lenovo 440-16i. Those you might get a lot cheaper than the original Broadcom versions. Before buying read up on the SlimSAS low profile cables you'll need for the Dell versions.
However, as you can see from the datasheets - I haven't measured it yet - the newer 9400 and especially 9500 series HBAs use a lot less power than previous generations, so if you can upgrade your card, you'll perhaps save more power than what you would get from deeper C states - which you then also get if it is so designed.
9500-8i, 6W typical
9400-8i, 10W typical
9300-8i, 13W typical
Of course the max value is quite a bit higher. So newer chips are more efficient and I've stopped buying the older chipsets a few years ago.
EDIT:
The newer 9500 series chips, i.e. SAS3808 and SAS3816, are also found in, for example, Lenovo and Dell HBA cards.
I think it is the Dell HBA350i, Dell HBA355i, Lenovo 440-8i and Lenovo 440-16i. Those you might get a lot cheaper than the original Broadcom versions. Before buying read up on the SlimSAS low profile cables you'll need for the Dell versions.
Last edited:
@john389 Thank you for the offer to test. I scoured the internet and could not find anyone who had examined the 9500-8i c-state. I actually have a very low performance use case of 12 3.5" hdds. When trying to research this topic I saw numerous reports that SAS2008/2308/3008 could add 30w to idle power. At my power rates that would be ~40 dollars per year. It was conceivable to me that I could breakeven with a 9500 card over several years if it idled considerably better. Without data I thought the 9500 card to be too risky to spend the extra money on the chance it idled better, so I ended up purchasing a SAS2008 chipset card.
To my surprise my system is still idling fairly well, C7 for ~88% of the time. Idle power went from 35 to 44w when I added the hba which appears to be from the card itself and then a few extra watts for random cycles that wake up the cpu. This is a new board to replace an old E5-2660v2 system and I have not swapped it out and connected it to the backplane yet though so hopefully adding the drives does not change the idle situation too much.
Power consumption seems to be a complicated thing to get information on online, there is so much variation from board to board and different platforms. This is with LGA 1700, 12th gen intel on debian 12.
If you still feel like sharing your results I would think folks would be interested, like I said I had never seen the 9500 reported and many of the power obsessed NAS builders shun HBAs entirely, going to sata add-in cards instead. I think as more 9400 and 9500 cards hit the second hand market there will be more interest from nas builders from power consumption. To me the SAS2008 and 3808 chips both appear viable low power nas cards.
To my surprise my system is still idling fairly well, C7 for ~88% of the time. Idle power went from 35 to 44w when I added the hba which appears to be from the card itself and then a few extra watts for random cycles that wake up the cpu. This is a new board to replace an old E5-2660v2 system and I have not swapped it out and connected it to the backplane yet though so hopefully adding the drives does not change the idle situation too much.
Power consumption seems to be a complicated thing to get information on online, there is so much variation from board to board and different platforms. This is with LGA 1700, 12th gen intel on debian 12.
If you still feel like sharing your results I would think folks would be interested, like I said I had never seen the 9500 reported and many of the power obsessed NAS builders shun HBAs entirely, going to sata add-in cards instead. I think as more 9400 and 9500 cards hit the second hand market there will be more interest from nas builders from power consumption. To me the SAS2008 and 3808 chips both appear viable low power nas cards.
That's because those things are non-related
Bullshit, I have each of mentioned generations plus 3408 and 3816 and none of them impacts the CPU ability to demote C states
Last edited:
They are.
Numerous drivers & cards (not necessarily LSI) are known to prevent deep C-States, e.g. due to the lack of ASPM
Ok, I double checked and I have ASPM disabled system-wide in BIOS, so this have nothing to do with CPU ability to demote cores into C6. What's next assumption?
Save your passive aggressive replies for someone else.
Yes, I can confirm that I also have a ConnectX-3 card installed in this system and the card itself supports ASPM L0s. You can see that SAS3408 (LSI 9400) also supports both ASPM L0s and L1. Even the Fusion ioMemory SX350 that doesn't support ASPM at all cannot change the behavior of the CPU's power saving - it reaches both C6 and PC6 states nice and easy.
The original question was about the hardware LSI chipsets limitation, not some rare OS handicaps. Here is the answer: LSI SAS chipsets (SAS2008 and newer) do support ASPM and has nothing to do with CPU ability to switch between C-states
Hey guys, I was doing research on this card and was wondering about the performance of NVMe drives on this card -- How does the read/write speed looks like comparing to direct PCIe connection?
I am asking because I also saw another thread saying the performance is poor (700MB/s read) from this thread https://forums.servethehome.com/ind...ance-on-broadcom-9440-8i-lenovo-530-8i.23534/
I am asking because I also saw another thread saying the performance is poor (700MB/s read) from this thread https://forums.servethehome.com/ind...ance-on-broadcom-9440-8i-lenovo-530-8i.23534/
Yeah, apparently they present NVMe devices as SCSI devices, negating half the advantage of NVMe. Sort of like AHCI PCIe SSDs from the early days of PCIe SSDs a decade ago. Tri-mode and U.3 is nothing short of a farce by Broadcom and Microchip to sell absurdly expensive crap to be able to save a little bit of cash on backplanes versus U.2.
