AMD Epyc 7443P optimal memory configuration

[littleprinze]

I'm configuring an system with an AMD Epyc 7443P with 256GB of memory. I'd prefer to use only 4 DIMMS so I can expand to 512GB in the future (motherboard has a max of 8 slots).

But Epyc chips have 8-channel memory so I'm wondering how much performance I would lose, also specifically with the AMD Epyc 7443P since this is a part with only 4 chiplets?

Very curious if any of you have experience and/or insights with this trade-off?

 

[alex_stief]

You leave anywhere between 0%...50% of performance on the table. Depends on what you do.
Most of the time when people don't specify what they do, the performance loss is closer to 0%
The 4 chiplets on the 7443P are enough to saturate all 8 memory channels for reads, so you can't just ignore it either.

 

[unwind-protect]

I have not seen benchmarks of EPYC 4-channel versus 8-channel. Outside of cryptography it is hard to imagine that it would make a noticeable difference.

You can test your workload on an existing machine going from dual-channel (or whatever you have) to 1-channel RAM and comparing.

 

[ari2asem]

populate your dimms according the manual of your motherboard. just easy.

for performance doesn't matter if you use 1R or 2R memory stick. or what cpu you use (CCX or CCD....doesn't matter)

if you want full performance, use 8 sticks. that's all.

 

[Spartus]

I'm in simulation engineering (CFD, FEA, etc.) and memory bandwidth is incredibly important. Many other workloads it's less so and 24 cores : 4 channels isn't awful for many typical workloads. I suspect things like video encoding and compiling code are more modest in their requirement, but I say that without great experience or data to pull upon.

of course you wouldn't really need 512GB for those workloads, so can you tell us what your main workload is?

 

[littleprinze]

Thank you for the thoughtful replies.

De server will be used as database server with with a sub-TB active dataset. The database is custom, but is similar to LMDB

Additionally the server is serving binary files referenced that are referenced from database records +/- 10TB

Everything is stored on ZFS.

 

[Spartus]

If you are chasing maximum number of small queries per second, yeah that bandwidth might matter, but if you are just caching a large database with low to modest query count then no it doesn't matter. Serving large files definitely won't make any difference.

 

[nk215]

Please post your build (CPU+MB+memory). I am looking into that same CPU +Tyan S8030GM2NE MB.

 

[ari2asem]

https://www.servethehome.com/amd-epyc-naples-memory-population-performance-impact-at-16-cores/

https://www.servethehome.com/amd-epyc-7002-rome-cpus-with-half-memory-bandwidth/

 

[hrgiger]

I have similar setup 7713p + 8 x 32GB 2R 3200Mhz, quite happy with performance my local runs matches with the published benchmarks, but motherboard got 16 slot so i was comfortable with going 32GB and I didnt need more, maybe instead you might reconsider motherboard?

*Also bios allows you to configure numa channels,deactivating cores, smt etc... so following amd memory population guideline and database tunning guide for the bios settings you might still get satisfactory results, ask also local supplier while you get a quote

 

[ari2asem]

AMD EPYC Performance Tuning - Thomas-Krenn-Wiki-en

Systems with AMD EPYC 7002 Rome CPUs can be optimized for performance in several areas. In this article you will find documents for optimizing operating system settings, BIOS settings as well as software configurations.

www.thomas-krenn.com

search on google with this and you will find some interesseting documents

amd high performance epyc documents

 

[ErikAnd]

For the cpu2007 benchmark spec.org two equal system except for the memory configuration are compared. The cpu is a EPYC 7313p. The first system is with 4 DIMMs and the second with 8 DIMMs. See:

CPU2017 Floating Point Rate Result: Supermicro A+ Server 1114S-WN10RT (H12SSW-NTR , AMD EPYC 7313P)

CFP2017 result for A+ Server 1114S-WN10RT (H12SSW-NTR , AMD EPYC 7313P); SPECrate2017_fp_base: 110; SPECrate2017_fp_peak: 120

www.spec.org

and

CPU2017 Floating Point Rate Result: Supermicro A+ Server 1114S-WN10RT (H12SSW-NTR , AMD EPYC 7313P)

CFP2017 result for A+ Server 1114S-WN10RT (H12SSW-NTR , AMD EPYC 7313P); SPECrate2017_fp_base: 176; SPECrate2017_fp_peak: 178

www.spec.org

For the bwaves testcase the difference is largest. The system with 4 DIMMs has a performance that is just 36% of the system with 8 DIMMs. Less than half performance! See yourself.