What motherboard supports the highest number of U.2/U.3 drives?

[fatherboard]

It turns out to achieve my in-machine optimistic storage goals I need a motherboard that can handle the utmost number of physical drives internally natively (no NAS).

What epyc motherboard supports the highest number of U.2/U.3 drive?

(Trying to keep each question under its relevant section)

 

[RolloZ170]

U.2 drives are usualy not connected directly to server motherboards, they use NVME Backplanes (with sensors) instead.
i do not know any motherboard where you can plug a U.2 drive on, always needs cables or backplanes.

 

[USER02744]

This [Supermicro board]( H14DSG-O-CPU | Motherboards | Products | Supermicro ) supports 20x PCIe 5.0 MCIO connectors, with usually can connect 2x U.2/U.3 drives or such to each. So that would give you 40x u.2 drives directly connected to the motherboard.

 

[ca3y6]

With PCIe switches you can achieve even more. You can connect 8 drives on a PCIe x16slots and 4 drives on a x8. On a H12SSL-i that gives you like 48 drives, plus another 2 via SlimSAS, and 2 M.2. Your power consumption will go through the roof with that many U.2 drives. In fact you might need to use some forms of backplanes, you would overload SATA power splitters. If you want lots of drives you should consider SAS, or even SATA SSD if you don't need the individual performance. SATA SSD have a fraction of the power consumption, and if put in parallel will achieve decent sequential speeds.

 

[fatherboard]

With PCIe switches you can achieve even more. You can connect 8 drives on a PCIe x16slots and 4 drives on a x8. On a H12SSL-i that gives you like 48 drives, plus another 2 via SlimSAS, and 2 M.2.

good to know, but I thought each U.2 takes 4 lanes, so PCIe x16 would accommodate 4, not 8 drives. Or did I get it wrong?
And how would I connect the U.2s to the PCIe x16 slots? is it with a u.2 to pcie 4 adapter card such as the GLOTRENDS PU41 Quad U.2 SSD to PCIe 4.0 X16 Adapter? or did you mean some cable adaptersfor PCIe x16 and x8, and which ones?

 

[i386]

In my experience mainboards with >16 pcie x4 port use a proprietary formfactor for specific chassis.

ASRock Rack GENOAD8X-2T/BCM is the only mainboard that has a ton of pcie slots for retimer/redriver cards...

good to know, but I thought each U.2 takes 4 lanes, so PCIe x16 would accommodate 4, not 8 drives. Or did I get it wrong?

U.2/3 ssds can have ports configured 1 x4 or 2 x2 for high availability or multipath. With pcie switches it's possible to connect a lot more pcie devices. Depending on the switches the performance can vary.

Here's a block diagram from supermicros jbof and how it uses pcie switches:

source:

SSG-136R-4MU32JBF | 1U | SuperServer | Products | Supermicro

1U front-loading all-flash storage box with 32 hot-swap 2.5" NVMe bays and PCIe 4.0

www.supermicro.com

 

[RolloZ170]

good to know, but I thought each U.2 takes 4 lanes, so PCIe x16 would accommodate 4, not 8 drives. Or did I get it wrong?

not native
PCIe Card with switches (like Ethernet Switch/Hub)

 

[nereith]

LR-Link lrnv9f96-10i - each SlimSAS 8i port on the board supports 2 U.2 SSDs, for a total of 20.

Since the card feeds into a PCIE 4.0 x16 slot, you get max 256GT/s aggregate throughput from the drives connected to that card.

You can always connect that card into an electrically narrower slot like x8 or x4, and/or use multiple copies of the card to connect more SSDs.

 

[ca3y6]

There is a whole thread on PCIe switches. Most are PCIe 3, which is still plenty of bandwidth: https://forums.servethehome.com/ind...apters-that-do-not-require-bifurcation.31172/

The bandwidth is shared between the devices but surely you will not use 40 U.2 drives in parallel at full bandwidth.

It seems to me that you seem to go in the direction of a custom build. U.2 cables are thick and having dozens in a box will be completely unmanageable. What can help is those PCIe3/SAS3 backplanes, which take an AUX power (there are GPU to AUX adapters) and allows for the force of all that cabling to not apply to the drive connector directly. Pardon Our Interruption...

 

[fatherboard]

H14DSG-O-CPU is definitely interesting, my dream is one that also has 48 RAM slots, then the party would be complete, everything inside one machine, because I'm worried 24 RAM slots is not enough.

 

[USER02744]

Not Sure that DDR5 spec would allow enough space to fit 24 RAM slots/CPU. I think the trace lengths would be too long without some VERY unique design.

 

[RolloZ170]

Not Sure that DDR5 spec would allow enough space to fit 24 RAM slots/CPU. I think the trace lengths would be too long without some VERY unique design.

why not ? clock is always reduced in 2DPC configs.

H13SSF | Motherboards | Products | Supermicro

 

[USER02744]

why not ? clock is always reduced in 2DPC configs.

H13SSF | Motherboards | Products | Supermicro

Perhaps I am reading this wrong - Signals and Connections for DDR5 Interfaces • Versal Adaptive SoC PCB Design User Guide (UG863) • Reader • AMD Technical Information Portal

But it seems that the recommended guideline is under 4000 mils, or 4 inches for DIMM interfaces? I guess I am assuming that is for running at the intended speed/design.

 

[RolloZ170]

But it seems that the recommended guideline is under 4000 mils, or 4 inches for DIMM interfaces?

yes, 1 mil is inch/1000

 

[SnJ9MX]

H14DSG-O-CPU is definitely interesting, my dream is one that also has 48 RAM slots, then the party would be complete, everything inside one machine, because I'm worried 24 RAM slots is not enough.

From the specs you linked, surely you aren't serious that 6TB of memory isn't enough.

Max Memory (1DPC): Up to 6TB