AMD EPYC Build - Memory Questions

Discussion in 'Processors and Motherboards' started by Erick, Nov 19, 2017.

  1. Erick

    Erick New Member

    Joined:
    Nov 16, 2017
    Messages:
    28
    Likes Received:
    19
    Hello all,

    I'm new here, and I'm looking to build a home server using the AMD EPYC platform. I have decided to go with an EPYC 7401P and a Gigabyte MZ31-AR0 board. It's been about 10 years since I last built an enterprise grade server, but I am very fluent with desktop components. I am having a hard time understanding how to populate the memory on this board to get the best performance I can, with a minimum of 64 GB of memory.

    As I understand, the faster the memory, the better with these new EPYC chips. Can I get by with using 2 X 32 GB RDIMM's @ 2666 mHz? I am looking into this kit: Crucial 64GB Kit (2 x 32GB) DDR4-2666 RDIMM CT11010536 | MZ31-AR0 | Crucial.com

    Here is the quick reference for the board: http://download.gigabyte.us/FileList/Manual/server_qrg_mz31-ar0_e_v10.pdf

    From what I can tell, using a pair of dual ranked RDIMM's would drop my speed down to 2400 mHz. Any insight would be greatly appreciated!

    Thank you,

    Erick
     
    #1
    Subsonic likes this.
  2. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,951
    Likes Received:
    860
    @Erick you really want 8 DIMMs. 8x16GB
     
    #2
    Greg C and Patrick like this.
  3. Erick

    Erick New Member

    Joined:
    Nov 16, 2017
    Messages:
    28
    Likes Received:
    19
    @MiniKnight So you are saying 8 DIMMs because it's up to 8 channels? What would the effective speed of the RAM be then?
     
    #3
  4. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,560
    Likes Received:
    4,490
    You can run 8x DIMMs at 2666. I would strongly recommend this as well. With two DIMMs you will have poor memory bandwidth
     
    #4
    Subsonic likes this.
  5. Erick

    Erick New Member

    Joined:
    Nov 16, 2017
    Messages:
    28
    Likes Received:
    19
    Yes I want to do whatever I need to do to get the maximum memory performance possible. So, I will go with 8x DIMMs @ 2666 MHz.

    Can anyone comment on if these will work for the maximum 2666 speed on the Gigabyte board listed above? Crucial 16GB DDR4-2666 RDIMM | CT16G4RFS4266 | Crucial.com If so, I will pick up 8 of these.

    Based on the documentation I should be able to achieve the full 2666 MHz, if I am reading it correctly. These are single ranked DIMMs.
     
    #5
  6. alex_stief

    alex_stief Active Member

    Joined:
    May 31, 2016
    Messages:
    417
    Likes Received:
    107
    Since it is single-rank, they should run at DDR4-2666 if the run.
    Epyc officially supports RDIMM DDR4-2666 single-rank and DDR4-2400 dual-rank.
    But I am still waiting for benchmarks single vs. dual. The benchmarks for Ryzen CPUs suggest that 2400 dual-rank delivers better performance than 2666 single-rank. And since it is basically the same CPU and memory controller, just 4 of them, I suspect that the results for Epyc would be similar.
    If I had to buy new memory for my dual-Epyc build, I would probably choose DDR4-2666 dual-rank based on the little information I found so far. Then either try to run it at DDR4-2666 or wait for a few more bios-updates that should increase memory compatibility. Until then I would run the memory at DDR4-2400.
    STH would really do us a favor testing single- vs. dual-rank memory on Epyc ;)
     
    #6
    Last edited: Nov 20, 2017
  7. Erick

    Erick New Member

    Joined:
    Nov 16, 2017
    Messages:
    28
    Likes Received:
    19
    I agree, there is very little information out there and I have probably read through everything there is on single vs dual rank with Ryzen. There are a lot of unknowns. I understand that the potential for better performance exists with dual rank but it is more difficult to run at higher speeds. Performance gains on Ryzen appear to be minimal, in the 1% to 3% range, but maybe this increases with more cores. Don't really know as there are no test results out there I can find.

    I think if price was equal, it would be a no brainer to go with dual rank, but I can get some single rank Crucial ram with the same specs as dual rank Crucial ram for ~25% less cost, so I might go that route, and use that extra $450 on SSD's. This is "just" a home server, something I'm going to be using for moderate video encoding tasks, nothing mission critical.
     
    #7
  8. alex_stief

    alex_stief Active Member

    Joined:
    May 31, 2016
    Messages:
    417
    Likes Received:
    107
    If $450 is 25% saved that means you are paying a total of $1800 for 8x16GB DDR4.
    Even with todays ridiculous prices I can find 16GB DDR4-2666 reg ECC (single or dual-rank) for ~180€. € with taxes in Germany is usually the same as US$.
    Is memory even more expensive in the US?
     
    #8
    Last edited: Nov 20, 2017
  9. Erick

    Erick New Member

    Joined:
    Nov 16, 2017
    Messages:
    28
    Likes Received:
    19
    Alex, Yes the cheapest dual rank 8x16 GB DDR4 I could find is roughly $1800 USD after tax. If you know of anything cheaper out there, I'm all ears :)
     
    #9
  10. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,560
    Likes Received:
    4,490
    Good point on SR v. DR RAM. Generally 1DPC you want DR with EPYC.

    And for someone searching this thread later, the reason you do not want 2 DIMMs per socket on the AMD EPYC is that the AMD EPYC package is four die. Each die has dual channel memory. If you have two DIMMs that means that two die will have half of their memory channels full while the other two will have no attached RAM.

    Always use 8 DIMMs with AMD EPYC. I have been told SR v. DR is measurable but not huge.
     
    #10
  11. alex_stief

    alex_stief Active Member

    Joined:
    May 31, 2016
    Messages:
    417
    Likes Received:
    107
    Ryzen benchmarks suggest that dual-rank vs. single-rank increases performance even slightly more than DDR4-2666 vs. DDR4-2400. In other words: Ryzen with dual-rank DDR4-2400 performs similar or slightly better than Ryzen with single-rank DDR4-2666 across a number of various benchmarks.
    Since you emphasized the importance of using DDR4-2666 instead of DDR4-2400 both here and in the reviews, I think this is worth further investigation.
     
    #11
  12. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,560
    Likes Received:
    4,490
    The bigger impact on EPYC is that Infinity Fabric works at RAM speed. So when you have multiple NUMA nodes die-to-die transfers are a bigger deal than RAM throughput. That impacts memory access across die as well as PCIe/ disk/ network access. That is not as big of an issue on Ryzen which is why I would treat the two differently in this regard.
     
    #12
  13. alex_stief

    alex_stief Active Member

    Joined:
    May 31, 2016
    Messages:
    417
    Likes Received:
    107
    I am with you so far in theory.
    But I won't be satisfied until I see some actual benchmarks ;)
    Because there may be a lot of applications where inter-die transfers are not a bottleneck.
     
    #13
  14. Erick

    Erick New Member

    Joined:
    Nov 16, 2017
    Messages:
    28
    Likes Received:
    19
    I was able to find some dual rank memory for a price similar with the single rank stuff. Same speed @2666 mHz.

    So, go with the dual rank? :)
     
    #14
    Patrick likes this.
  15. bash

    bash Member

    Joined:
    Dec 14, 2015
    Messages:
    91
    Likes Received:
    21
  16. alex_stief

    alex_stief Active Member

    Joined:
    May 31, 2016
    Messages:
    417
    Likes Received:
    107
    Epyc has an 8-channel memory interface. With only 4 DIMMs, you are wasting performance in bandwidth-bound applications.
     
    #16
Similar Threads: EPYC Build
Forum Title Date
Processors and Motherboards AMD Epyc ROME Memory Population Performance - 4 or 8 RDIMMs? Friday at 9:42 PM
Processors and Motherboards Epyc 3000 platforms beyond mini-ITX Dec 1, 2019
Processors and Motherboards Possible to run EPYC gen1 in gen2 boards? Nov 20, 2019
Processors and Motherboards Epyc Rome, L3 Cache and CCX-es and CCDs Nov 18, 2019
Processors and Motherboards Reduce Epyc idle power consumption? Nov 15, 2019

Share This Page