Looking for a 24i HBA that plays nice with ZFS

jbrukardt

Member
Feb 4, 2016
74
35
18
88
I'm migrating away from my RAID6 array, which I have currently running on a Areca 1883ix-24 with miniSAS cables going out to a NORCO 4220 with backplane version SAS 3505 version 3.5.

This is a passive backplane, not a SAS multiplier.

What recommendations would you all recommend for a SAS controller to control 24 SAS3 10tb drives. These days I don't need RAID anymore (or want it, since it messes with ZFS, unraid, etc).

I am also reasonably open to two controllers if necessary, but no more.

I understand that my Areca can be put in JBOD mode, but i hear that even in JBOD mode it breaks ZFS
 

MichalPL

Member
Feb 10, 2019
99
11
8
For 16x 10TB SATA few years ago the winner was LSI 9206-16e. 4.8GB/s real zfs transfers (ssd sata) on HDD slightly below 4GB/s sequencial (~2.1GB/s real case).

So my reccommendation (if the card is not too old and just pcie 3.0 and 16x 6gbps) is 2 x LSI 9206-16e (yes mess/tricks with cables becouse of "e"), much faster than expanders plus card.

Btw. What budget and expected speed?
 
Last edited:

nasbdh9

Active Member
Aug 4, 2019
107
39
28
Any 12G 8i + 82885T/82885T*2, can easily get 24/48 disk ports
this configuration does not have any performance bottleneck for HDD
 

i386

Well-Known Member
Mar 18, 2016
3,237
1,043
113
33
Germany
Broadcom and Microchip have new sas 24gbit/s hbas/raid Controller with Up to 32 Ports and pcie 4.0 x16 Host Interfaces
 

mobilenvidia

Moderator
Sep 25, 2011
1,954
198
63
New Zealand
Broadcom and Microchip have new sas 24gbit/s hbas/raid Controller with Up to 32 Ports and pcie 4.0 x16 Host Interfaces
As seen on sticky post I updated earlier today, well almost, LSI based

SAS3916 8GB, PCIe 4.0, 12Gbs SAS, CacheVault
  • Lenovo RAID 940-32i 8GB + Expander
SAS4024 eHBA, PCIe 4.0, 24Gbs SAS
  • LSI SAS 9600-24i
  • LSI SAS 9600-24e
SAS4116 PCIe 4.0, 24Gbs SAS, CacheVault
  • LSI SAS 9660-16i
  • LSI SAS 9670w-16i 16x PCIe4.0 bus
SAS4124 PCIe 4.0, 24Gbs SAS, CacheVault
  • LSI SAS 9670-24i
 

MichalPL

Member
Feb 10, 2019
99
11
8
wow they are really fast :) btw Is it beneficial to use 24gbps if hdd is HGST 10TB metal? My old (back then super ultra modern) hdd was able to do 280MB/s without filesystem (250MB/s on zfs)
 

mobilenvidia

Moderator
Sep 25, 2011
1,954
198
63
New Zealand
Just showing what bragging rights deep pockets can bring, but do you need deep pockets ?

In the end a 250MB/s HDD only just maxes out a 3GBps SAS1/SATA2 port
5x HDD's = 10Gbe saturated
World is full of bottlenecks
 

barichardson

New Member
Mar 31, 2022
5
5
3
I've been happy with an LSI 9305-24i and it plays nice with ZFS. Supports SAS3 speeds and doesn't cost as much as the newer tri-mode cards. Just make sure to update the firmware to the latest version (16.00.12.00) since it fixes some SAS bus reset issues.

I also designed a fan shroud that can be 3d printed to keep it cool since 24i HBAs get very hot:
LSI 9305-24i fan shroud by barichardson
 
  • Like
Reactions: Stephan

jbrukardt

Member
Feb 4, 2016
74
35
18
88
Any 12G 8i + 82885T/82885T*2, can easily get 24/48 disk ports
this configuration does not have any performance bottleneck for HDD
ended up going with this solution

LSI 9300-16i + 82885T for what should be a total of 28 disk capability.


Sanity check my plan?

9300-16i

Port 0 - Direct attach to disks 0-3
Port 1 - Direct attach to disks 4-7
Port 2 - Connect to expander port G
Port 3 - Connect to expander port F

82885T

Port A - Direct connect to disks 8-11
Port B - Direct connect to disk 12-15
Port C - Direct connect to disk 16-19
Port D - Future direct to disk capacity
Port E - Future direct to disk capacity
Port F - Connected to 9300-16i port 2
Port G - Connected to 9300-16i port 3
Port H - These appear to only put input capable from another HBA or Expander?
Port I - These appear to only put input capable from another HBA or Expander?


Hows all that look? I would like to have a little more future capacity, but it looks like the external ports cannot do direct to disk eh?
 

nasbdh9

Active Member
Aug 4, 2019
107
39
28
9300-16i only needs to be connected to G port of 82885T, so there is already 48Gbps bandwidth between 82885T and 9300-16i, enough for 24 HDDs to load 2Gbps at same time
If you already have a 9300-16i, the best solution is to connect four 82885T cards to maximize the number of ports
but note however that the total bandwidth is still limited by PCIe 3.0 x8

Ports H and I are generally used to connect external JBOD
 

nasbdh9

Active Member
Aug 4, 2019
107
39
28
82885T must update the latest firmware, it solves a lot of interchange/connection bug compared to old firmware
 

Stereodude

Active Member
Feb 21, 2016
428
72
28
USA
4 lanes of SAS3 is good for >4GB/sec. Some of the expanders can connect to SATA3 HDDs at 6gig speeds and still connect at 12gig on the back end. PCIe 3.0 x8 is good for about 6.4GB/sec. Old cheap cards have 8 ports of SAS3 and are PCIe 3.0 x8. 6GB/sec hardly seems slow even by today's standards.