Supermicro Chassis or Power Supply to GPUs

[9jack9]

@funkywizard Thanks for your detailed post a while back! I stumbled across this thread via google as I'm trying to figure out what GPUs I can realistically power with a CSE-846 case. The tip for using the 4-pin connectors on the CPUs was top notch (provided they were not greater than 115W TDP) altered my build plan, so thanks for sharing.

I don't imagine the "good" PDU (PDB-PT826-8824) is compatible with the CSE-846 (4U) enclosure? Also, can you provide guidance regarding the amount of GPU power I can expect to provide via PDB-PT747-4648 on 120V?

Specifically, I am looking to run a Quadro RTX 4000 (Max Power Consumption 160W) and a second Nvidia GPU (not sure which yet, but likely a higher end GeForce card). Also looking to run a SQ model power supply (as this guy isn't going in a basement exactly).

 

[funkywizard]

@funkywizard Thanks for your detailed post a while back! I stumbled across this thread via google as I'm trying to figure out what GPUs I can realistically power with a CSE-846 case. The tip for using the 4-pin connectors on the CPUs was top notch (provided they were not greater than 115W TDP) altered my build plan, so thanks for sharing.

I don't imagine the "good" PDU (PDB-PT826-8824) is compatible with the CSE-846 (4U) enclosure? Also, can you provide guidance regarding the amount of GPU power I can expect to provide via PDB-PT747-4648 on 120V?

Specifically, I am looking to run a Quadro RTX 4000 (Max Power Consumption 160W) and a second Nvidia GPU (not sure which yet, but likely a higher end GeForce card). Also looking to run a SQ model power supply (as this guy isn't going in a basement exactly).

Glad to have helped!

"I don't imagine the "good" PDU (PDB-PT826-8824) is compatible with the CSE-846 (4U) enclosure"

PDB-PT826-8824 -- appears to be a 2u power distrbution board, from a quick google search. So, in that case, no, it won't be compatible.

If you don't mind a ghetto server build, it's easier to just get a 2u 825 or 826 chassis and replace the PDB (if it's an "825 model" PDB) with an 826 PDB. Then remove the back plate / pcie retention off. Then you can slot in gpus without the 2u retention getting in the way (goes without saying the lid needs to be off too). This is better for airflow too, as a GPU on the far side of the chassis won't have it's airflow blocked by the full 4u side of the chassis. Even when I got 4u chassis set up with the "better" 4u PDB, it was more of a challenge cooling GPUs in the 4u case vs the 2u case.

The result is not pretty, but works. Also, can't stack anything on top of it for obvious reasons, and it's effectively an "open air case" at that point -- not ideal if someone might accidentally bump into it or short it out.

Regarding 120v power, definitely the lower end PDBs have a slightly harder time supplying enough power to the GPU when using 120v power. To get around this slightly, use both power supplies at the same time, the higher wattage supplies the better. I forget the exact details, but, it was something like, with 2x 1600w psus on 208v power, I c0uld power 2x 1080ti using a very low quality PDB, versus, say, 1x 1080ti with 1x 1280w psu on 120v power. Whereas with a better quality PDB, I could power 3x 1080ti using 1x 1280w power supply on 120v.

The 825 2u PDB is just junk, while the 826 is quite usable. The 4u PDBs I've had a hard time finding any that are good, but some are worse than others.

 

[9jack9]

If I didn't need the hard drive slots, I would consider the 2U case ...but at this rate I would likely just use a standard ATX power supply (like the EVGA 1600 Platinum), and gut out what is in the enclosure to begin with.

The 4u PDBs I've had a hard time finding any that are good, but some are worse than others.

In this regard, can you specify which PDUs have been worse than others? In terms of GPU support I can't tell the difference between PDB-PT846-2824 and what ships with the enclosure, PDB-PT848-8824.

 

[StevenDTX]

Some more details:


First, I take the 2x4pin connector, and connect 4 pins to one motherboard 8 pin connector, and the other 4 pins to the second 8 pin motherboard connector.

Based on what I could find online, everything I read said this would definitely be fine for CPUs <100W TDP, probably OK for 115w TDP CPUs, and probably NOT OK for 130w+ CPUs or overclocked CPUs. I've had no issues with 115w TDP Intel chips (E5-2670v1 / E5-2680v2) with this configuration.

That leaves you with an available 8-pin EPS plug coming off the PDB. This can be converted to a 2x 8-pin PCIe cable.
Supermicro part number CBL-0424L -- I've bought these from wiredzone.com or superbiiz.com. I've used quite a few of these and they work great.

superbiiz.com: CBL-0424L Supermicro 40cm 8Pin to Dual 6+2Pin GPU Power Extension Cable - Internal Power Cable - SuperBiiz.com
wiredzone.com: Supermicro CBL-0424L Power Cables

Looks like both are out of stock right now, but they usually have them.

I would say, expect to power no more than 8+6 off the 8+8 cable. One exception is the Vega, as I draw quite a bit less than 200 watts on those, even though it has 2x 8-pin PCIe, this works fine. 1080ti has 1x 8-pin + 1x 6-pin. That has worked fine as well. However, powering 2 1080ti's, using both 8-pins on this to power the 8-pins on each of 2x 1080ti, and then connecting up something else for the 6-pins on those 1080ti's, I've not had that work out, so don't expect it to. I don't know if I've tried powering two cards that each need an 8-pin connector -- it might work, but don't count on it.

So that gets you part of the way there. Works fine for 1-gpu servers.


==

Thank you very much for this! I have an 836, with dual E5-2680 v2's and I wanted to add an Nvidia RTX 4000. Based on your post, I was able to use one 8-pin EPS connector for one CPU, and the 4-pin for the other CPU. I was then able to use the second 8-pin to power the GPU.

I found the SuperMicro cable on ebay, and it will be here next week. But, I ended up finding this on Amazon yesterday and it was delivered today. It worked perfectly!

https://www.amazon.com/gp/product/B07C31TZCW