so when you say that it needs DDR3 cassettes, you mean that you can use the ones out of the DL580 G8 that came with v2 CPU's correct?lol now it's super cost effective server for 1400EUR + 1TB DDR3 next ~1500 lol, but need cassettes for DDR3 with older memory chip
DDR3 is also +150W (for 64 dimms), ok it's late - end of "hobby" time
Xeon E7 8894 v4 - what type of Memory for best performance (single core)?
- Thread starter MichalPL
- Start date
-
- Tags
- 8894v4 dl580 gen9 ram ddr4 xeon
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Yes - here I used ram cassetes from G8, I dont know if ddr3 cassettes exist with the new chip (3200MT/s) - then there will be no sense to use DDR4 at all
Final thoughts on the memory in HP580 G9.
Just bought another 512GB DDR4 and test it, now its 1TB DDR4 2133MHz @ 1600MHz (32x32 DIMMs) CL12:
So the final result is: it doesn't matter what type of memory you will use, it can be even DDR3 not DDR4, just use 8 same DIMMs per CPU (architecture is kind of "octa channel" per CPU).
Difference between DDR3 and DDR4 is minimal (~1%), single core performance same on DDR3 (1333MHz CL9) and DDR4 (1600MHz CL12).
Ram make bigger difference when not all slots populated (then you see ~1.5% max between Rank 2 or 4 only in muli-core tests), but when done right any type of memory will perform almost same and better than any Epyc CPU (not for the single core performance - also TR5995X is better in almost everything ).
Just bought another 512GB DDR4 and test it, now its 1TB DDR4 2133MHz @ 1600MHz (32x32 DIMMs) CL12:
So the final result is: it doesn't matter what type of memory you will use, it can be even DDR3 not DDR4, just use 8 same DIMMs per CPU (architecture is kind of "octa channel" per CPU).
Difference between DDR3 and DDR4 is minimal (~1%), single core performance same on DDR3 (1333MHz CL9) and DDR4 (1600MHz CL12).
Ram make bigger difference when not all slots populated (then you see ~1.5% max between Rank 2 or 4 only in muli-core tests), but when done right any type of memory will perform almost same and better than any Epyc CPU
(not for the single core performance - also TR5995X is better in almost everything ).
Here's a few caviar's:
Typically server motherboards have daisy chain memory layout. (same goes to most current amd zen desktop mobos)
The fastest configuration is to have 1 stick per channel, adding more sticks adds capacity but increases latency to access data by few ns on each refresh/tick/clock cycle. The added memory sticks will increase latency but in some cases it will provide greater total bandwidth (not everywhere software level, and not on everything hardware).
Same rules apply on the sticks themselves, more ranks or banks often mean more latency. (depends on motherboard, and type of workload.)
In most cases a single rank 4bit provide best latency in servers. They are faster, but are not as dense.
They will suffer from lesser total bandwidth at cost of much better latency.
Lesser amount of banks = less latency, but potentially lower total bandwidth - unless chips are fast enough.
Load Reduce chips increase latency a lot, but also increase capacity and potential total bandwidth.
Registered mem increase latency, but it provides lesser of a hit when working in daisy chain scenarios vs normal unbuffered memory.
There's no point of using DDR4 clocked at lower Hz than 2133MHz, unless you are going for capacity.
The IO bus clock, as well as memory chip clock is often higher with higher clocked ram, resulting in overall better system performance.
(The DDR3 modules clocked at same speeds as DDR4 can have less latency, but much less capacity.)
Typically server motherboards have daisy chain memory layout. (same goes to most current amd zen desktop mobos)
The fastest configuration is to have 1 stick per channel, adding more sticks adds capacity but increases latency to access data by few ns on each refresh/tick/clock cycle. The added memory sticks will increase latency but in some cases it will provide greater total bandwidth (not everywhere software level, and not on everything hardware).
Same rules apply on the sticks themselves, more ranks or banks often mean more latency. (depends on motherboard, and type of workload.)
In most cases a single rank 4bit provide best latency in servers. They are faster, but are not as dense.
They will suffer from lesser total bandwidth at cost of much better latency.
Lesser amount of banks = less latency, but potentially lower total bandwidth - unless chips are fast enough.
Load Reduce chips increase latency a lot, but also increase capacity and potential total bandwidth.
Registered mem increase latency, but it provides lesser of a hit when working in daisy chain scenarios vs normal unbuffered memory.
There's no point of using DDR4 clocked at lower Hz than 2133MHz, unless you are going for capacity.
The IO bus clock, as well as memory chip clock is often higher with higher clocked ram, resulting in overall better system performance.
(The DDR3 modules clocked at same speeds as DDR4 can have less latency, but much less capacity.)
It only applies in a very limited fashion to this situation, because E7 v4 use external memory buffers (e.g. Jordan Creek)
typical Registered memory's are two or more DIMMs in once with only one DIMM(address)load.
one 2666Mt 2Rx? RDIMM works with 2666Mt
the equvalent two DIMMs 2666Mt 1Rx? slow down to 2400Mt (POR)
sure unbuffered memory ist faster, but limited in GB size.
imagen a board with 8 UDIMMs per channel.
In HP DL580 "G9" it was from ~320W up to ~1200W
1TB 64x16GB DDR3 1866MHz @ 1600MHz
4x E7 8894v4 (4x 24 cores)
Last edited:
You are super close
E7 v2/v3/v4 support 24 DIMMs per CPU.
so it is "6 DIMMs" per CPU channel
or "3 DIMMs" per Jordan Creek mem buffer chip channel
Not possible here :/ (possible on E5 v2 even on DDR3 -1680v2 also possible with E5 v3, I think up to 2600 or slightly more when overclocked - 1660v3/1680v3).
Exactly, I spend some time to tweak it, and it's different than any other Xeons.
How it works and why 2400MHz is not possible:
E7 v2/v3/v4 chips has 4 memory channels connected to the memory buffer - Jordan Creek or Jordan Creek 2.
E7 v2 has 4 lanes of SMI working at 2666MHz or 1600MHz
E7 v3/v4 has 4 lanes of SMI2 working at 3200MHz or 1866MHz (or 2666/1600MHz when connected to older Jordan Creek).
each Jordan Creek chip have 2 mem channels and can work at:
1333MHz (2666MHz to the CPU) or in a single channel mode 1600MHz (1600MHz to the CPU)
Jordan Creek 2: 1600MHz or 1866MHz.
So fastest possible configuration for v3/v4 is:
E7 v4
*(SMI2 ch1)Jordan Creek @ 3200MHz
#(DDR ch1)DDR3 @ 1600MHz
#(DDR ch2)DDR3 @ 1600MHz
*(SMI2 ch2)Jordan Creek @ 3200MHz
#(DDR ch3)DDR3 @ 1600MHz
#(DDR ch4)DDR3 @ 1600MHz
*(SMI2 ch3)Jordan Creek @ 3200MHz
#(DDR ch5)DDR3 @ 1600MHz
#(DDR ch6)DDR3 @ 1600MHz
*(SMI2 ch4)Jordan Creek @ 3200MHz
#(DDR ch7)DDR3 @ 1600MHz
#(DDR ch8)DDR3 @ 1600MHz
it is much faster than:
E7 v4
*(SMI2 ch1)Jordan Creek @ 1866MHz
#(DDR ch1)DDR3 @ 1866MHz
*(SMI2 ch2)Jordan Creek @ 1866MHz
#(DDR ch3)DDR3 @ 1866MHz
*(SMI2 ch3)Jordan Creek @ 1866MHz
#(DDR ch5)DDR3 @ 1866MHz
*(SMI2 ch4)Jordan Creek @ 1866MHz
#(DDR ch7)DDR3 @ 1866MHz
SMI is working at 3200MHz, in time when DDR4 was ~2400MHz, so it's similar to 9000MHz today
I was not able to find any mode or solution faster than 1866MHz and this 1866MHz was not optimal, and much slower than 2x1600MHz.