Epyc Genoa Build Advice

[Le_Potate]

Hello All,



I'm putting together a build based on Epyc Genoa specifically for use with Ansys HFSS, and secondarily large body structural and cfd. If there is a better sub-forum for this, please feel free to move it.



Current parts list:

Epyc 9654

128Gb RDIMM @4800Mt/s x12

Asus ESC4000A-E12-26WGP

RTX 6000 ada x2

Samsung PM9A3 x2

Sabrent Rocket 5 2tb x4

For OS, 50/50 Windows 10 for Workstation, or RHEL 8.



The goal is to run high level simulations, and was curious if anyone here has had the opportunity to play with EPYC and Ansys. Also, I've read some conflicting arguments about memory bandwidth between Genoa and Sapphire rapids. Am I missing something though? Is this the best platform to go with?

I am a technician that works at a field site for a research university. While we aren't out in the wilderness or anything, accessing the proper IT resources is cumbersome and sometimes just flat out unavailable. So a few of us pitch in for hardware projects. I have a decent understanding of what I'm doing, but not an expert's.

Thanks in advance

 

[Le_Potate]

Seems like this should be the right place?

 

[Le_Potate]

Alright. Well, forums are clearly not the correct venue for this. Third different forum ive posted on with zero reply. Feel free to delete. I'll just go **** off to somewhere else

 

[RolloZ170]

do you expect years experience from servetheHOME users ?
even Genoa is very new, same than sapphire rapids.
we are still in progress to learn whats going on with that new platforms.

 

[ano]

you want the 12 ramsticks with genoa like you suggest, cpu, drives etc are good (the pm9a3)

 

[SnJ9MX]

What does Ansys say about running Ansys on EPYC Genoa? I'm assuming you've already checked it's compatible. They'll likely have recommended hardware. Why have you chosen server hardware instead of a more workstation oriented platform (Threadripper)? Threadripper is 20-25% faster for both single thread and multithread.

AMD EPYC 9654 vs AMD Ryzen Threadripper PRO 7995WX [cpubenchmark.net] by PassMark Software

www.cpubenchmark.net

 

[Bert]

There is no easy answer here; no one can guess which platform would be better without testing it and understanding the hardware requirements of the software. I would actually ask the software provider, they should run their software on target hardware platforms and provide a recommendation.

For example does this software scale up for 192 cores?

 

[rtech]

Get it from store which has generous return policy if something does not work out return it
You could also check out EPYC tuning guide on amd.com maybe theres is some info to be scrounged there.

 

[bayleyw]

hard to make suggestions for a $40k build with no well defined workload profile. my generic suggestions:

(1) epyc is not a workstation platform. if you're buying a $40k system with someone elses's money, I'd suggest an oem TR PRO system with a return policy and vendor support.

(2) profile your workload before you commit. this is doubly true of epyc which has an ok-but-occasionally-wonky internal layout due to the chiplets.

(3) sapphire rapids has fewer memory channels, but better vector instruction support which might be useful for HPC codes

(4) those gpus are fp32 only - are you sure they are what you want for HPC? maybe some A100's are in your future

(5) a 2U server capable of taking 4x GPU is going to be rather loud

I think the most important part here is you are spending a huge chunk of cash that doesn't belong to you. unless you have utmost confidence that your build is optimal, you should buy a vendor approved workstation so you can blame the vendor when things don't work out. Ansys is notoriously fickle when it comes to scaling, you can find random gremlins like explicit dynamics (which only scales to six cores) or CFD, which is 100% bandwidth bound