Multi-NVMe (m.2, u.2) adapters that do not require bifurcation

[zachj]

Unless you need the bandwidth of dedicated lanes for the NVMe drives I personally like the OWC U2 Shuttle.

I’ve got two of them and that gives me 8 m.2 slots across 8 lanes of PCIe. I’ve got them connected to m.2 slots on my motherboard so I don’t actually take up any PCIe slots, but you can certainly install them in a PCIe->U.2 adapter too

 

[eazysnatch]

I'm not sure if you're referring to my case, but I'm searching for something entirely different. I need a PCI with dual or quad NVME.

I don't need "Bifurcation," which means a cheaper card ( my BIOS supports Bifurcation ). The downside of that setup is that it's not hot pluggable.
Here is an example: AOC-SLG3-2M2 | Add-on Cards | Accessories | Products - Super Micro Computer, Inc.

 

[nexox]

I don't need "Bifurcation," which means a cheaper card ( my BIOS supports Bifurcation )

In that case you're in the wrong thread, this one is for adapters which do not require BIOS bifurcation, so only adapters which include a PCIe switch.

 

[eazysnatch]

You are correct, sorry about that. Thank you for the heads up.

 

[luckylinux]

No I am talking about U.2 HGST SSDs connected with cables, they do not get powered by the card, they get powered by a sata/molex connector. Also tried through an intel backplane (connected to the caecent card) which gets powered by an ATX connector (picture below). So I don't think the problem is power. And also those HGST drives work fine on the same system using PCIe bifurcation.

But yes, cooling is a challenge. I resolved it with having big noctua fans blowing directly into the drives, keeps them under 45 deg celcius even with the fan at low speed.

View attachment 41402

Did you have some 3D Print Model for that Intel U.2 to Oculink SFF-8611 Backplane by the Way ? I just ordered a few of those Backplanes and I'm now looking for some [hopefully not so expensive] SFF-8611 Cables. But I thought for the Desktop / Server in a Desktop Chassis, it would be good to have something holding the Drives and the Backplane together other than Zip Ties .

 

[ca3y6]

Did you have some 3D Print Model for that Intel U.2 to Oculink SFF-8611 Backplane by the Way ?

See attached (STL + openscad file). Won't win any prize but will do the job. Need to print with something that will sustain heat (eg ABS/ASA). I also leave a big noctua fan on top of the drives to keep them cool.

 

[pimposh]

What a niche solution... yet nice to see. Gen5x16, 8x M.2 (Gen5x4), Areca ARC-1689-8N based on PEX89048 for those who do not dare to click the link.

 

[luckylinux]

See attached (STL + openscad file). Won't win any prize but will do the job. Need to print with something that will sustain heat (eg ABS/ASA). I also leave a big noctua fan on top of the drives to keep them cool.

Super, Thanks. I will send it to Craftcloud for Printing. Usually I just go with PETG 40% Infill .

 

[ca3y6]

Sorry to come back to this, but do you also have the "outer" Enclosure or "Braces" ?

Or at least there seem to be some "Braces" that hold them at a constant Separation Distance:
View attachment 45083

That's the STL file I shared. You print two of them.

 

[ca3y6]

Correct. Yeah my file naming skills are somewhat lame. I use similar braces with different spacing for large arrays of 7mm SATA SSD (holding them in blocks of 8 or 12). Right now I am not in array building mood anymore but that will inevitable come back. My next step will be to learn how to make my own custom SATA power cables as it was the biggest pain in my butt!

 

[luckylinux]

What a niche solution... yet nice to see. Gen5x16, 8x M.2 (Gen5x4), Areca ARC-1689-8N based on PEX89048 for those who do not dare to click the link.

I can already imagine the Price without even clicking

 

[ca3y6]

That model wouldn't work, I need a much greater density of SSDs, so the space between disks is around 4-5mm. I currently use this kind of splitter, where the cables are coming straight in the direction of the SATA socket as opposed to perpendicular. But then they take a lot of space, and are often a source of electrical problems.

 

[luckylinux]

See attached (STL + openscad file). Won't win any prize but will do the job. Need to print with something that will sustain heat (eg ABS/ASA). I also leave a big noctua fan on top of the drives to keep them cool.

Sorry to come back to this, but do you also have the "outer" Enclosure or "Braces" ?

Or at least there seem to be some "Braces" that hold them at a constant Separation Distance:

 

[ca3y6]

That’s for 7mm SATA SSD. For u.2 I use the backplane pictured above as the cables are otherwise too thick and it’s a mess.

I use those high density SATA SSD arrays in two places, first in the free space between the MB and the native backplane inside a 2u chassis in a datacentre, and second in a home NAS, where low noise level is critical, so a server chassis wouldn’t work. Plus server backplanes are designed for 15mm drives, so are less dense than what I need (and also not great from an airflow point if view, require powerful fans).

My home NAS had 40 such SATA SSD at one point, before I realised I was paying for electricity for a flash array that was essentially empty. So I have now half of that in storage.

 

[tubeit]

For u.2 I use the backplane pictured above

Mind sharing the model? I've read the whole thread but I could find a reference to the model (hopefully I did not miss it..)

 

[luckylinux]

That's the STL file I shared. You print two of them.

Aaah it goes in the other Direction


When I read "Caddy" in the Filename I was thinking something you insert into the Backplane (you know, like 2.5'' Caddies or 3.5'' Caddies), not a holder where you screw into the SSD directly from the Side.

So I just need 2x of these for each Backplane, that's all ?

 

[ca3y6]

Mind sharing the model? I've read the whole thread but I could find a reference to the model (hopefully I did not miss it..)

yeah it was on a different thread: Intel 8-Bay PCIe SAS/SATA Backplane Board P/N: H88386-251 Tested Working | eBay UK

Does SAS and PCIe3, and is powered by an ATX power connector. There are GPU/PCIe power to ATX converters if you don't have a spare one. I used that COMeap PCIe 8 Pin to CPU 8 Pin (4+4) EPS-12V Adapter, Black, Male-to-Female, Motherboard Power Cable, Compatible with Motherboards, 13.3 inches, 12V DC, High Speed: Amazon.co.uk: Computers & Accessories

 

[tubeit]

Damn, there's nothing similar on european eBay. I will have to look for another model (I'm still considering if backplane is the way to go..)

 

[luckylinux]

Correct. Yeah my file naming skills are somewhat lame. I use similar braces with different spacing for large arrays of 7mm SATA SSD (holding them in blocks of 8 or 12). Right now I am not in array building mood anymore but that will inevitable come back. My next step will be to learn how to make my own custom SATA power cables as it was the biggest pain in my butt!

Why do you do that ? I assume Cost is the main Issue ...

Something like Silverstone SST-PP07-BTSB should be quite reliable IMHO:

Silverstone SST-PP07-BTSB - 4-pin Molex to 4x SATA 30cm Extension - Black: Amazon.de: PC & Video Games

Silverstone SST-PP07-BTSB - 4-pin Molex to 4x SATA 30cm Extension - Black: Amazon.de: PC & Video Games

www.amazon.de

 

[ca3y6]

You can use some parcel forwarding service like forward2me. The backplanes are fairly cheap, won't cost you much in taxes to import them.

 

[TRACKER]

Damn, there's nothing similar on european eBay. I will have to look for another model (I'm still considering if backplane is the way to go..)

ou yes there is
sorry about the inconvenience

 

[luckylinux]

That model wouldn't work, I need a much greater density of SSDs, so the space between disks is around 4-5mm. I currently use this kind of splitter, where the cables are coming straight in the direction of the SATA socket as opposed to perpendicular. But then they take a lot of space, and are often a source of electrical problems.

For SATA or U.2 ? If U.2 I guess you'll have a bigger Cooling (not to mention Money) Issue.

Why not something like a Supermicro SC216 at that Point ? You could also get a (arguably expensive at ~ 100-150 EUR) NVMe Backplane for it for 24 x 2.5'' SAS/SATA or NVMe SSDs.

 

[tubeit]

ou yes there is
sorry about the inconvenience

Thanks, got one more option now.
Although with shipping and import duties (thanks Brexit) it doesn't cost much less than this chassis with the backplane in it Intel R2000 19" Server Gehäuse mit 8-fach 12G SFF Backplane & Rails // Rr2000 | eBay
Will have to consider.. at these prices I assume there are more backplane options (for example the backplane that I have in my G292-Z20)

 

[nexox]

Will have to consider.. at these prices I assume there are more backplane options

I've looked for quite a while at ebay backplane options and so far haven't come up with anything better for U.2 than the Intel, everything else either uses proprietary connectors, fits a proprietary chassis, or is some combination of rare and expensive. I'm tempted to grab another kit from the UK seller since they're in stock again, even though I haven't installed the last one I bought yet.

 

[TRACKER]

I saw that one but it's MUCH more expensive than what @ca3y6 proposes. Like 3 x the Cost.

And make sure it's the NVMe one. The one I saw a couple Weeks back was SATA/SAS. The NVMe one was out of Stock IIRC.

It is NVME (even though only SAS/SATA is mentioned) because i bought one and it works flawlessly with my p4610 1.6T U.2 drives (via oculink)

 

[tubeit]

Make sure you plan your way through, it may use "standard" components in terms of e.g. PSU and Risers, but do your own Research before pulling the Trigger.

At best you could slide in a Supermicro ~400-600W PSU with Cables in that Format (1U / Redundant), if it fits.

At worst you need to buy a Proprietary PDU (Power Distribution Unit) and 1-2 x PSUs with "Golden Finger" Type Connectors (that plug into the PDU, which does the Redundancy).

Definitely, not buying more stuff soon unless there's a clear plan, I'm still considering my options and if it's worth doing anything at all.

I'd love to find a cheap server that can house at least 10-12 NVMe/SAS/SATA and has a decently recent CPU (like Zen2-3) with support for 2 or 3 PCI-E cards, that idles at ~150W. But apart from some 1st/2nd gen Xeon Scalable servers, there's not much interesting stuff on the low end from what I can see.

Story time

I thought that the G292-Z20 could be what I needed, but honestly the idle consumption is too high for doing not much at all. Also, only the mezzanine cards support PCI-E bifurcation so I have my M.2 drives on the outflow stream of hot air coming from the CPU, which is part of the reason why I'm migrating to U.2/SATA in the backplane in the front.
I've 3d printed an intake and an extractor to have all the central fans run slower and this thing screams quite a lot anyway (the CPU still runs hot, 55-60° at 90-100W idle).
Overall not impressed with this chassis, I think it's good for people who load it with GPUs and don't care about power consumption, but it's definitely not my endgame.

 

[kapone]

I'd love to find a cheap server that can house at least 10-12 NVMe/SAS/SATA and has a decently recent CPU (like Zen2-3) with support for 2 or 3 PCI-E cards, that idles at ~150W. But apart from some 1st/2nd gen Xeon Scalable servers, there's not much interesting stuff on the low end from what I can see.

- cheap...
- at least 10-12 NVMe/SAS/SATA
- recent CPU (like Zen2-3)
- 2 or 3 PCI-E cards

er...that's most likely not going to happen. I think you may have to "settle" for a 3647 based system, or even older. I wouldn't worry about the idle power consumption of any system from the last 10-12 years, they're pretty similar. In fact the newer systems consume more power at idle...yes, they're more powerful, but unless you're utilizing them fully, it's just wasted compute.