Mismatched channel capacities on Epyc

bayleyw

Active Member
Jan 8, 2014
189
65
28
Due to a shipping mishap I wound up with mismatched DIMMs in my EPYC system (5x32GB + 3x64GB). How does interleaving work in this case? Will the bottom 32GB of each channel be 8-way interleaved or am I completely screwed?
 

RolloZ170

Well-Known Member
Apr 24, 2016
2,572
643
113
55
which Mainboard do you have ?
if you have a Dell Epyc system:

4.3 Identical CPU and DIMM Parts Identical DIMMs must be used across all DIMM slots (i.e. same Dell part number).
Dell EMC does not support DIMM mixing in Rome systems.
This means that only one rank, speed, capacity and DIMM type shall exist within the system.
This principle applies to the processors as well; multi-socket Rome systems shall be populated with identical CPUs
 

bayleyw

Active Member
Jan 8, 2014
189
65
28
I have a Supermicro board. I installed the RAM and ran some benchmarks, the usual suspects ran fine, but of course, the piece of scientific computing code I actually needed to run ran four times (!) slower with the mismatched memory. To be fair, it also ran four times slower with just channels ACEG populated so I think something else is going on there...
 

alex_stief

Well-Known Member
May 31, 2016
833
286
63
37
If you are doing anything remotely sensitive to memory bandwidth/latency, do not make any compromises with balanced memory population. That goes for pretty much every platform, but especially for AMD Epyc. A lot of stuff described as "scientific computing" falls into this camp.
 

RolloZ170

Well-Known Member
Apr 24, 2016
2,572
643
113
55
To be fair, it also ran four times slower with just channels ACEG populated so I think something else is going on there...
what have yo expected ? with interleave on the upper 96GB is three channel RAM.
without interleave you have separate single channels.
at Rome 6 channels give you NOT the expected 75% bandwidth, this is corrected(optimized) in Milan.
please change to 8x 32GB(or 64GB)
 

bayleyw

Active Member
Jan 8, 2014
189
65
28
Well I would expect a moderate performance drop going from 8 channels to 4, but not to the level I saw. Makes we wonder whether Supermicro is doing something wonky with interleaving with anything less than 8 channels (the BIOS options are 'Auto' and 'Disabled').
In any case I'll move to 8x32, leave the 3x64 for some other project, and put this down as an example of why you shouldn't get creative with your memory population.
 

RolloZ170

Well-Known Member
Apr 24, 2016
2,572
643
113
55
Makes we wonder whether Supermicro is doing something wonky with interleaving with anything less than 8 channels
its AMI BIOS. the interleaving is AMD stuff. pls make some Memory-bandwidth tests with AIDA64.
remember in the worst case you run with only 3 DIMMs(the upper or lower 96GB depending how the mapping of the OS works)
AMD recommends (strict) balanced memory, think about. don't blame supermicro without testing gigabyte/asrock rack first, thanks.