Gen4 U.2 NVMe Drives - How to connect to motherboard?

[ectoplasmosis]

I have several Asrock ROMED8-2T EPYC Milan based systems, and wish to add 4x Samsung PM9A3 Gen4 U.2 SSDs to them.

The boards have 2x 4-lane Oculink connectors, which will work fine for two of the drives using Oculink SFF-8611 to U.2 SFF-8639 cables.

How about connecting the other two at Gen4 speeds?

Are there any passive PCIe cards which break out a 8x/16x slot into 2/4 Oculink connectors using bifurcation, which are compatible with Gen4 drives and speeds?

Would using something like this: HYPER M.2 X16 GEN 4 CARD｜Motherboards｜ASUS Global
...with two of these: Delock Produkte 62984 M.2 Key M zu U.2 SFF-8639 NVMe Adapter mit 50 cm Kabel
...work at Gen4?

 

[BlueFox]

You could also use the M.2 to U.2 adapters with the M.2 slots on the board. You will lose the #2 PCIe slot, but there are plenty more.

 

[ectoplasmosis]

You could also use the M.2 to U.2 adapters with the M.2 slots on the board. You will lose the #2 PCIe slot, but there are plenty more.

Ah, forgot to mention, the M.2 slots are already full.

It would be ideal if there was a passive PCIe 16x card to 4x Oculink adapter that supports Gen4 speeds. Does such a thing exist?

 

[uldise]

Does such a thing exist?

i don't think so.. for Gen4 SlimSas is all the way..

 

[ectoplasmosis]

i don't think so.. for Gen4 SlimSas is all the way..

Great! Are there passive SlimSAS Gen4 PCIe adapter cards out there?

 

[uldise]

this one? AOC-SLG4-4E4T | Add-on Cards | Accessories | Products - Super Micro Computer, Inc.
i don't know if this is what you mean "passive"..

 

[ectoplasmosis]

this one? AOC-SLG4-4E4T | Add-on Cards | Accessories | Products - Super Micro Computer, Inc.
i don't know if this is what you mean "passive"..

Thanks!

Seems to be unavailable to purchase anywhere though, and ships with a half-height slot plate.

By ‘passive’, I mean no host controller or PCIe multiplexing chips, just direct traces to the 16x lanes on the slot with the motherboard bifurcating into 2x8 or 4x4.

 

[uldise]

Seems to be unavailable to purchase anywhere though, and ships with a half-height slot plate.

try to find some SM retailer and ask for it, it should be available.. and i think it comes with both brackets. and yes, this one will work with motherboard bifurcating.

 

[mirrormax]

the m.2 adapter+those delock cables would work and might be the cheapest alternative until pcie 4.0 breakout boards become more common, you can get a broadcom 9500-8i but thats like 500$+. do you really need that pcie4 speed on all 4 drives? you should still get max IOPS with pcie3.0 for most workloads

2 of these might be cheapest option actually Amazon.com: Ableconn PEXU2-132 NVMe 2.5-inch U.2 (SFF-8639) SSD PCIe x4 Carrier Adapter Card - Support Intel 750 2.5-inch U.2 SFF SSD: Computers & Accessories
or these with cables https://www.amazon.com/Ableconn-PEX-OL153-OCuLink-SFF-8612-Adapter/dp/B08DR17F5F?ref_=ast_sto_dp

 

[uldise]

if you can live with gen3, i just finished similar setup with gen3 oculink card, see here: https://forums.servethehome.com/ind...ct-boards-notes-experiences.33468/post-310770

 

[ectoplasmosis]

Great, thanks for the tips!

Gen4 speeds are essential; this is to go into a storage server for ingesting uncompressed real-time video, so sequential performance is critical and I already own 8x Samsung PM9A3 Gen4 drives.

Cost is secondary to being able to bifurcate a single 16-lane slot into 4x 4-lane NVMe channels.

Going to look into a Broadcom P411W-32P as well as the Supermicro card.

 

[uldise]

Broadcom P411W-32P

this is not 'passive' as it have PEX88048 PCIe Gen 4.0 switch chip.

P411W-32P

Enable Enterprise Class x4 NVMe Acceleration and Caching Solutions

www.broadcom.com

 

[UhClem]

... Gen4 speeds are essential; this is to go into a storage server for ingesting uncompressed real-time video, so sequential performance is critical and I already own 8x Samsung PM9A3 Gen4 drives.

Are you sure? [ie, essential?] Exactly which video format are you slurping?
[It's important to know your needs; then you can plan/allow for future/expansion. Nevertheless, since you already have both a Gen4 host & Gen4 targets, it IS highly desirable to get the full benefits.]

Cost is secondary to being able to bifurcate a single 16-lane slot into 4x 4-lane NVMe channels.

Going to look into a Broadcom P411W-32P as well as the Supermicro card.

The beauty of a passive (*furcation) solution, in addition to the low cost, is that, unlike switches and retimers, it is not definitively limited to a specific PCIe Gen level (or lower). There is a strong likelihood that a simple and straightforward 4x-x4 SFF8643 (NVMe) adapter like [Link] combined with standard SFF8643-SFF8639 cables will work just fine for you (delivering full Gen4). For inspiration ( ), see https://forums.servethehome.com/index.php?threads/pcie3-nvme-cable-length-connections-retimer.33659/. Give it a shot--only ~$35 [I've bought (different cards) from Ceacent/Ali twice--work great--delivery ~14 days.]

[Moving from the ghetto to a private estate ...] That Broadcom P411W-32P card looks very nice. Thanks for pointing it out. PCIe switches are cool!

 

[BlueFox]

If you weren't aware, the PM9A3 is not really any faster than PCIe 3.0 devices when it comes to sequential writes. It's rated for 3.5GB/s. Plenty of high end PCIe 3.0 ones are similar (Intel P4610 is 3.2GB/s for example).

 

[ectoplasmosis]

If you weren't aware, the PM9A3 is not really any faster than PCIe 3.0 devices when it comes to sequential writes. It's rated for 3.5GB/s. Plenty of high end PCIe 3.0 ones are similar (Intel P4610 is 3.2GB/s for example).

I appreciate this, but this storage server will be hammered with sustained sequential reads also, and this at Gen4, at least theoretically, should be much faster than Gen3. Besides, all components support Gen4, so why not?

 

[ectoplasmosis]

Are you sure? [ie, essential?] Exactly which video format are you slurping?

We are ingesting multiple streams of ProRes 4:4:4:4 at 4K50 (~500MB/s per stream), plus tons of NDI and potentially SMPTE 2110 in the future (1.5GB/s per stream), along with simultaneously serving the content to editors over 10Gbit SFP+ Thunderbolt 3 adapters.

I'll get hold of a few cheap & cheerful bifurcation breakout cards and see what happens while I wait for something like the Supermicro adapter to become available.

 

[NateS]

Check out this thread; it links to several passive options:

https://forums.servethehome.com/index.php?threads/pcie-gen-4-bifurcation-risers-and-nvme-ssds.32943/

Keep in mind that your cable/trace length limits are much shorter on Gen4, and that includes the length of traces on the MB to get to the CPU, so retimer-based options may be required depending on your physical system layout. Unlike a switch chip, a retimer adds much less latency and I believe is lower power, so if your motherboard supports bifurcation, and you need some extra length, that's the way to go.