Super Micro EPYC / NVidia thermal approved systems

[Jason Hirsch]

Good Morning-

I'm having a heck of a time finding a thermal approved nvidia/EPYC system. Specifically I was looking at Super Micro and, when going through the configurations, was flat out told there'd be no warranty if I put nvidia cards into that EPYC box.

The 'hush' was that Intel and NVidia are in bed to such an extent that no one would warranty a system with threadripper chips and Quadros or TIs. I'm aware of the recent changes to the tensorflow API legaleze.

So are you aware of a vendor approved unit that can have AMD and nvidia chips playing nice- and one that I won't get soaked on a $400k purchase by voiding the warranty?

Thank you

 

[Patrick]

Threadripper or EPYC? I am meeting with some EPYC systems PMs today and can ask.

How many GPUs? What models?

 

[Myth]

We build servers with AMD and Nvidia for the media and entertainment industry.

 

[i386]

We build servers with AMD and Nvidia for the media and entertainment industry.

What's the name or website of your company?

 

[Jason Hirsch]

EPYC. Current configuration was 2x 20core Intel chips. Apparently I can get an dual EPYC system with 2 full slot PCIEs, but that would be about it. I need to push in some fast networking to offload the data when done, and some very fast pcie intel 4600 SSDs to process the data since our code isn't clean or optimized.

I apologize for using epyc/threadripper interchangeably. I got sloppy on the phone.

I'm just really surprised there isn't a certified case /mother board combo. Usually supermicro is right out there.

 

[Jason Hirsch]

Threadripper or EPYC? I am meeting with some EPYC systems PMs today and can ask.

How many GPUs? What models?

Original plan was to push in P5000 or 1080tis and upgrade later. Ideally given the infancy of the codebase I'd put in 2x 1080TIs. In addition to that, 2x Intel 4600 PCIE drive, and a 2xQSFP NIC... but still needing space free. That works out to be just under 1.2kw.

 

[Patrick]

Usually the SM Ultra servers can support 3x GPUs, including active, plus a few more add-in cards and storage. https://www.servethehome.com/nvidia-gpu-in-a-amd-epyc-server-tips-for-tensorflow-and-cryptomining/

 

[Jason Hirsch]

Usually the SM Ultra servers can support 3x GPUs, including active, plus a few more add-in cards and storage. https://www.servethehome.com/nvidia-gpu-in-a-amd-epyc-server-tips-for-tensorflow-and-cryptomining/

Worked my way through the thinkmate configuration for that and got pretty close. Just missing dual 10gbe nics.

These damn last minute engineering change requests are such a pain in the back.

 

[Blinky 42]

Worked my way through the thinkmate configuration for that and got pretty close. Just missing dual 10gbe nics.

Having used them all over the years - For ThinkMate (or Broadberry or ServersDirect /Equus) just call up their sales folks and say this is the config I am starting with but I want 10g <and anything else you want>, should be able to get somewhat better pricing than the list price in their online config tools as well. They can quote them out with the GPUs as well if that is easier for your accounting folks vs sourcing them on your own.
Note that the "Ultra" servers typically have a bit longer lead time than the non-ultra ones due to the way they get sourced from SuperMicro.

 

[Jason Hirsch]

Having used them all over the years - For ThinkMate (or Broadberry or ServersDirect /Equus) just call up their sales folks and say this is the config I am starting with but I want 10g <and anything else you want>, should be able to get somewhat better pricing than the list price in their online config tools as well. They can quote them out with the GPUs as well if that is easier for your accounting folks vs sourcing them on your own.
Note that the "Ultra" servers typically have a bit longer lead time than the non-ultra ones due to the way they get sourced from SuperMicro.

ThinkMate (Jon) has been awesome.

He helped me prove to the company they were already on the books. What a pain that was.

Nah whats been driving me wonkers is (first) finding out that there was no thermal GPU approved cases- so I couldn't grow. One of our engineers is all over the TensorFlow and I found him money. And, frankly, I'm sick of Intel doing naughty things to their prices. So throwing a little cash towards AMD is my little 'screw ya'- and it's even better they have a nice product!

But squeezing everything in there AND getting it past/approved by accounting has proven to be too hard.

I sent them off a couple of configs. Right now I'm held up because the engineers won't use/vet Server, so they want to run Windows 10 with 64 cores Yeah that's fun.

 

[Blinky 42]

ThinkMate (Jon) has been awesome.
...
I sent them off a couple of configs. Right now I'm held up because the engineers won't use/vet Server, so they want to run Windows 10 with 64 cores Yeah that's fun.

Heh good luck on that front. Battling against what people are used to but is a crap-tastic setup I have encountered far too often. Jon has been quite helpful to us in the past as well as well as their support folks on the Source Code side.

 

[Jason Hirsch]

Heh good luck on that front. Battling against what people are used to but is a crap-tastic setup I have encountered far too often. Jon has been quite helpful to us in the past as well as well as their support folks on the Source Code side.

Well, said coder is now my boss... so... "Yeah that's a great idea!"