Please help me pick a server or MB with lots of PCI-E.

[crabstorage]

I would like to set up a server for AI/inference. Already have some GPUs. An RX580 and a P40. I couldn't get the P40 running in any of my other systems due to incompatibility. I will eventually get more GPU as my budget allows. They will be high vram GPUs like more P40s or 3090, etc.

I was set on the Supermicro H11SSL-i or even the V2 of that board, but I'm afraid that it wont' support resizable BAR like my other desktop boards and the P40 will not work. So what about a complete used server? Any recommends for something that will enable me to install at least 3 or 4 GPU? My budget is $1000 and quiet or lower(ish) power consumption would be a plus. No worries though, it's going into my garage. I will add at least 128 or 256 of ram and shoot for 16+ cores.

Saw some options like Poweredge R720 and ML 350 G9 but I'm not super up on the server space anymore. The last ones I had were DL380 G3.

Of course if I can just build off some motherboard within the same budget I will do that. Asking for your expert recommends.

 

[ano]

h12ssl-i?

 

[crabstorage]

Will it work with the P40? And AFAIK it's more expensive but had PCIE4.

Like this: Supermicro H12SSL-i + AMD EPYC 7262+ 2133P 8GB * 2 ~ 32GB * 8 RAM | eBay is one of the only options and it's my entire budget without PS or case.

 

[vvkvvk]

H11 gen doesn't have BIOS checksum protection so the BIOS can be "hacked" with GitHub - xCuri0/ReBarUEFI: Resizable BAR for (almost) any UEFI system

There's someone here who has done it iirc.

E: List of working motherboards · Issue #11 · xCuri0/ReBarUEFI

 

[custom90gt]

I guess I don't know why a P40 wouldn't work, I have a P40 in my H12SSL-i and it works. All I use it for is plex and steam stream though so maybe that's why it's ok for me.

 

[ziggygt]

Some of the miner dedicated motherboards have a lot of GPU slots but strange form factors and limited memory. With multiple GPUs they are anything but quiet. GPUs need room. Some regular boards have the slots but they are too close to one another. Not sure if the miner MB can support the SW you need but might be worth taking a look. With the devaluation of crypto somethings might be a good deal?

 

[crabstorage]

So I am ok to hack H11SSL-i? I patched laptop bios and ryzen bios before but never a server. Is PCIE-4 vs PCIE3 worth it, you think for extra money? I want to buy the cheaper board but my thoughts of buy once, cry once are coming to the surface.

And with the devaluation of crypto.. nothing is changing. Inflation is making components go up.

 

[ziggygt]

That looks like a really nice MB. it has a slot for m.2 memory and has bifurcated PCIe slots that would enable multiple m.2 in a single PCIe slot. Good luck with your build.

 

[kpfleming]

PCI-E Gn4 vs. PCI-E Gen3 makes no difference at all unless you have cards that support PCI-E Gen4

 

[crabstorage]

For ML I think it won't make much difference currently, maybe for a parallel workload like training only. To buy the newer Supermicro board I have to up my budget.

Nobody made a stand alone server for this? It would alleviate the need for buying power supplies separate and finding a case.

 

[kpfleming]

Supermicro sells barebones systems based on their motherboards, you should be able to find a system based on an H11SSL-i. It's going be challenging to fit within your budget though.

 

[jdnz]

Nobody made a stand alone server for this? It would alleviate the need for buying power supplies separate and finding a case.

supermicro do a pile of GPU oriented systems - starting with the 7049GP and 740GP 'desktop' systems ( which support 4 enterprise grade gpus ) going up to monster systems

GPU Servers For AI, Deep / Machine Learning & HPC | Supermicro

Dive into Supermicro's GPU-accelerated servers, specifically engineered for AI, Machine Learning, and High-Performance Computing.

www.supermicro.com

just be aware that even the 7049GP/740GP are very loud - as they have to push enough air to keep fanless enterprise pcie GPUs cool ( we've got two 7049 and a 740 in use - all with 4 gpus, and I'm glad they're in our machine room and not in someone's office )

 

[crabstorage]

Thanks for the suggestion. I will see if any surplussed servers work. I assume generation is the same as the MB.. H11 and H12, etc.

Seems like PowerEdge C4130 is the cheapie non supermicro option. I think P40 and rtx3090 would work in it.. I think.

 

[Patriot]

Thanks for the suggestion. I will see if any surplussed servers work. I assume generation is the same as the MB.. H11 and H12, etc.

Seems like PowerEdge C4130 is the cheapie non supermicro option. I think P40 and rtx3090 would work in it.. I think.

P40 and 3090 are vastly different beasts... you are not going to fit a 3090 (unless blower) in a server chassis.

 

[crabstorage]

This is true. I am trying to price 4130 vs building the H11/H12 boards out. I can always take a grinder to the 4130 top cover or buy/construct a different type of cooler. Also have to see if the 3090 takes up 3 slots rather than 2, etc.

All in all this is going to be a fun adventure.

 

[crabstorage]

Ended up buying this, it was easier than fabricating a case with proper cooling and buying all the fans separately.

Supermicro 4028GR-TRT 4U GPU Barebone Sever w/ X10DRG-OT+-CPU w/ GPU Board | eBay

X9DRG-O-PCie, Supports up to 8 GPUs Via X9DRG-O-PCIE GPU Board. SUPERMICRO SERVERS. Processor X10DRG-OT+-CPU. Back Plane. System Board. Network Card. HP WORKSTATIONS. Rails No Rails Included.

www.ebay.com

Cpus:INTEL XEON E5-2683 V4 2.1 GHz 16-CORE SR2JT CM8066002023604 | eBay
Ram:Samsung M393A4K40CB1-CRC0Q 32GB 2Rx4 PC4-2400T DDR4-19200 CL17 288-Pin RDIMM | eBay

About $1600 total and no waiting from china.

 

[ziggygt]

Impressive, but I think you blew the budget.

 

[crabstorage]

I think I would break my budget with the bare board too. I would have to buy the case, fans and PS and pray they work and keep the server cool in a place with no a/c. I might have to buy fans and silence the server I did get, but it's not a priority right away. Now I will have to measure if this server will consume more than my home A/C unit

 

[ziggygt]

The more expensive route is to build a Frankenstein system and then replace it later. You have a beauty there. I have found that the power consumption numbers in IPMI are not very accurate. I plugged my system into a UPS monitored by another system and used that to measure the power consumption. To get a handle on the CPU draw I used CORETEMP with all the drives off it actually matched consumption pretty well.