How to connect 24 * Intel P4510 to the Broadcom 9600-24i HBA.

[chemist_slime]

Hello community, long time lurker but first time poster, thanks for all the amazing content!

I could not find anything online that was definitive so I'm hoping someone here can clarify. Would the Intel P4510 drives, which are U.2, connect through the Broadcom Cable 05-60006-00 cable - a SlimSAS 8i S‌FF-8654 to 8 X SFF-8639 U.3 Cable to the 9600-24i HBA. If not, are there any u.2 -> u.3 adapters out there? These drives are not going into an off the shelf server, instead they are going into Sliger NAS case which will house a mini-ITX motherboard (AsRock Rack X570D4I-2T), so I only have 1 PCIE 4.0 16x slot. Thanks in advance.

 

[SlowmoDK]

What connector are on the backplane in your Sliger Case ?

And does it support your U2/NVMe ?

 

[chemist_slime]

Hi @SlowmoDK ! This is the case: CX3701 | Sliger It's just a normal SATA-SATA backplane but I plan to remove it. It's a 10 bay 3.5" NAS which I plan to later upgrade to the 24 bay version (WIP) hence the 9600-24i HBA. For now I know it's going to become a bit of a frankenstein-nas but it only makes it more fun. I'm willing to string along adapters to make this work.

 

[T_Minus]

Why do you want 24x NVME if you're only going to use 1 HBA? Why not 20x SATA and then 4x NVME? WAY less power needs, way easier to connect, way cheaper to connect. Go with a supermicro SC216 with the 20+4 backplane, no DIY hacking together, no adding desktop PSU.

The CX3701 does not have the capacity to cool 8x 2.5" Enterprise NVME let alone 24.

Any HDD or SSD NAS not meant for NVME will most likely fail at cooling and keeping 2.5" ENterprise NVME operational, they will over heat.

 

[chemist_slime]

@T_Minus Please indulge me. I have a short depth rack, 15", so any off the shelf NAS is pretty much too deep and more importantly, way too loud. I currently have the Synology 1221+, which is an 8+4 (extension), 12" depth and it's just too slow. It has 8 spinners + 4 ssds.

 

[BlueFox]

~400W TDP for those SSDs alone (if you were using 25W ones, that would be 600W total). You will need multiple high static pressure fans to accommodate. Not happening in that case.

You would also need a backplane with PCIe switches with that HBA for that many NVMe drives. That's the only way to go beyond 6. They're all proprietary and case specific, so, also not happening in that case.

 

[twin_savage]

Would the Intel P4510 drives, which are U.2, connect through the Broadcom Cable 05-60006-00 cable - a SlimSAS 8i S‌FF-8654 to 8 X SFF-8639 U.3 Cable to the 9600-24i HBA.

I can confirm that they will not.
That being said, SlimSAS 8i S‌FF-8654 to 8 X SFF-8639 U.2 cables exist but I doubt the Broadcom HBA would be compatible with them since Broadcom doesn't offer the cable for sale themself.

 

[twin_savage]

You would also need a backplane with PCIe switches with that HBA for that many NVMe drives. That's the only way to go beyond 6.

This may be true for Broadcom, but hardware raid cards and even HBAs from more helpful vendors let you attach to NVMe disks with x1, x2 or x4 PCIe lanes per SSD without a backplane; just don't expect them to be passed through as completely native NVMe (some of the NVMe featureset is going to be stripped away).

 

[chemist_slime]

Thank you all for the suggestion/recommendation, would something like this work instead? U.3 (SFF-8639) PCIe Gen 4 16GT/s to M.2 NVMe SSD Adapter . It converts an M.2 to a U.3 interface while also maintaining pcie 4 speeds. The M.2 power consumption is also lower, at around 10W. I'm looking at the Corsair MP600 https://www.amazon.com/Corsair-MP60...600&qid=1706236999&s=electronics&sr=1-16&th=1 as the ssd.

 

[SlowmoDK]

Just go with the 10 sata or sas drives your case supports, that also matches your motherboard setup
If you need more than 4-6 NVMe's that will require a different platform (EPYC/XEON) and case

But also maybe give a description of that you are trying to accomplice with setup, along with a budget.

I'll hold of here, mainly because I get just a hint of trolling vibe from OP

 

[mattventura]

Hate to join the dogpile, but....

I don't know if the Broadcom Tri-Mode HBAs have improved much since the 94xx series, but the 94xx cards do not expose NVMe devices directly. They translate it and expose it to the system as a SAS drive, so even if you don't bottleneck it down to 1 lane by trying to run 24 of them on one card, the performance still won't be as good as native NVMe.

Either go with 24 SAS/SATA SSDs as others have said, or use NVMe drives connected natively (not through a tri-mode HBA) and just buy larger drives. You'll end up with better performance, lower power consumption, and a less expensive overall setup.

Edit: I realized I forgot one of the most obvious issues with the 24i - it still has an x8 host interface. So you're not just bottlenecking 96 lanes worth of SSDs down to 24 lanes - you're bottlenecking it down to 8. You'd get the same best-case throughput by using the two onboard OCuLink connectors on the board and not using an HBA at all!

 

[i386]

(Am i the only one WHO uses ssds for small and random Io? )

 

[chemist_slime]

Thanks for all the recommendations, I hear you all. In the end, this is what I've decided to do. I met Henry of ApexStorage at CES this year, great guy, and decided to just take the plunge and get the Apex x21: Apex Storage X21 NVMe AIC

It was a bit on the pricey side just for the controller but at least it scales to 21 nvme drives and saturates the full x16 4.0 lanes. The ASRock mini-itx has a single m.2 so I'll just use that for the boot drive. It also has 8 SATA ports so I'll then add the 8 spinners into the Sliger case. So in the end, it's x2 more expensive but I'll also get to fit x2 the amount of drives and fully utilize the system. It should also be able to keep things cool because I do agree, those P4510 do consume quite some watts.

Not sure why anyone would call me a troll. I'm simply trying to get a NAS that I can fit into a ventilated closet, be small, quiet and fast.

I'll post some benchmarks once everything is together if you guys are interested.

 

[BlueFox]

I think what many are trying to get at is that you've yet to detail what you are actually trying to accomplish. We can then provide recommendations based upon that. Why do you think you need so many NVMe SSDs? I believe most would be of the opinion that you just wasted $3k given the limitations of it and the system you're putting it in.

Also, you're still going to have cooling problems with that case and card...

 

[SlowmoDK]

If you are not trolling us ... OMG

Maybe pay a consultant some of the money you are wasting atm

 

[itronin]

I think what many are trying to get at is that you've yet to detail what you are actually trying to accomplish. We can then provide recommendations based upon that. Why do you think you need so many NVMe SSDs? I believe most would be of the opinion that you just wasted $3k given the limitations of it and the system you're putting it in.

Also, you're still going to have cooling problems with that case and card...

ISO's. I think @chemist_slime needs to store a bunch of ISO's.

In the beginning perhaps all those used high cap nvmes looked very attractive from a cost/storage standpoint and they started down this path?

Good luck to ya with your Sliger project. As others have offered, heat, heat, and more heat. I suggest you monitor your temps all the way through the chassis around these drives lest you find yourself with some expensive gum stick paperweights.

 

[chemist_slime]

Haha, you guys are hilarious. I'm a big fan of SFF and wfh. I simply cannot have a screaming server anywhere near me so this was the only option. I also have limited space so I tend to cram as much into 1 system to be efficient. It costs a bit more but this does allow me to expand as NVME drives become cheaper and higher in capacity. Trust me, I'd love to get a rack but it's not an option right now - I have a 15U wall mount, so...

But quick question, I'm a big fan of synology but ever since they started drive-locking I started looking elsewhere/DIY. Besides truenas, are there any other alternatives that come close to Synology's feature set? This NAS will serve as an NFS to other VMs that lie in another cluster as well as for general storage/backup. The mini-itx board also has dual 10G, which was a huge plus.

 

[itronin]

Haha, you guys are hilarious. I'm a big fan of SFF and wfh.
...
The mini-itx board also has dual 10G, which was a huge plus.

me too! Not rack-able but small, relatively quiet (soft whirr).

Not sure what board you are using but in mine the 10g, and the cpu produce an awful lot of heat as does the 9400-16i I'm using.

RE OS distributuions you could look at unraid - personally not my cup of tea esp for vm storage but its an option for an ecosystem (docker/container) oriented ecosystem. Scale has a catalog of apps.