DDR5 ECC RDIMM temps, especially in a workstation chassis

[compgeek89]

This is more of a memory question, but chassis seems like the closest relevant category!

I have just built and am testing a Threadripper 7000 build. Specs for reference:

AMD Ryzen Threadripper 7970X 350W SP6 - 32-Core/64-Threads

Asetek 35-102-0000354 836S-M1A AMD Threadripper Cooler 360mm

ASUS Pro WS TRX50-SAGE WIFI

NEMIX RAM 256GB (4X64GB) DDR5 5600MHZ PC5-44800 2Rx4 ECC RDIMM

Samsung 990 Pro 2TB

PNY RTX 4000 Ada VCNRTX4000ADA-PB 20GB 160-bit GDDR6

Fractal Design Define 7 XL ATX Full Tower Case

Super Flower LEADEX VII XG 1300 W 80+ Gold

Windows 11 Enterprise 64 Bit

Noctua A14 PWM x5

Noctua A14 FLX x1

Noctua NF-F12 x4

So, I have the case set up for airflow with the radiator top-mounted. The four NF-F12 fans are across the top, three on the radiator, one on the extra space (with the extra space being toward the back, radiator pushed to front). Then three A14s on the front, two on the bottom, one on the back. Air pushing front to back, bottom to top.

In general, temps are good, but the RAM is a little crazy. The motherboard has two RDIMM slots above the CPU and two below. After 24hrs of Prime95 "blend" the average temp for the top stick closest to the socket is 80C, the one above that is averaging upper 70s, but both have hit 95C. The bottom two are staying in the 60s average with a max around 80C. I know the designed operating range for the RAM is 0-95C, but is this OK? If not, is there anything I can/should do about it? I am not having stability issues. I ran the Windows extended memory test for almost 24hrs with no issues, and Prime95 has also been stable for 24hr runs.

Consumer RAM doesn't come anywhere close to this. I have read a few things about RDIMMs running hotter... but I can't find anything conclusive about what is acceptable. Obviously this is unlikely to be the continuous operating status for the machine in daily use. It's supposed to be a Solidworks FEA workstation for the long haul.

I appreciate any insight you guys might have.

 

[compgeek89]

P.S. Here are some reference shots at the 24hr mark:

 

[RolloZ170]

I have read a few things about ECC RAM running hotter...

its not the ECC but the buffer chip (the one in the middle, IDT or M(ontage) or Rambus)

 

[compgeek89]

its not the ECC but the buffer chip (the one in the middle, IDT or M(ontage) or Rambus)

Good correction. My question still stands, though!

Looking into things further, I think the 95.8C readings are glitches with HWInfo64 -- it's hanging in the 80s on a largefft prime95 test, and every once in a while there's a straight one sample spike to 95.8, doesn't seem like it's actually getting there. I'm not sure to what extent the end-user's workflow will ever get to these temps but ... is this reasonable? I guess it's within spec, but... seems crazy high.

 

[sam55todd]

For some reasons my regular non-overclocked single-rank DDR5 ECC RDIMM 16GB 4800 Micron
(MTC10F1084S1RC48BA1 NGCC EC8 RDIMM DDR5 1RX8 - PC5-4800B-RD0-1010-XT)
is flagged as hot by HWInfo64 even at temps above 35 ^C
(this is under normal load, not stress-testing/benchmarking)
despite Micron' specification clearly saying it has operational temperature up to 95 ^C


I've just received heatsink prototype components - 0.9mm aluminum sheets (alternatively 1.2mm because they don't have 1mm) and spring clamps
(or one of those but I'm not sure if high-carbon spring steel is foldable well enough without breaking or holds sufficient grip on heating to 90^C)
to install on memory buffer/ecc section after some folding/cutting effort since capacitors around it are 1mm higher (plus obviously thermal compound),
I hope it might improve situation a bit (although at these low temps chances are quite slim)
Unless I've received a weird batch of engineering sample memory from ebay hustlers of course.

 

[compgeek89]

This setup brough my temps down around 8C

 

[Thorvaldt]

I have a similar issue, well maybe the temps are a non issue if 95 celcius is allowed.
I have a Genoa EPYC 9634 system with 512GB of ddr ecc 4800 mem from Samsung, under heavy load some modules peak at around 78 celcius (in mid summer, temps in room about 30 degrees), while the 6 others stay around 65 degrees. The ones that hit 78, are located below the CPU, the two middle ones, furthers away from the top exhaust ventilators. The CPU hardly gets above 60 degrees nomatter what I throw at it, the big surface really helps.
Mainboard: Asrock GENOAD8X-2T/BCM, which is a humongous board that just about fits the Fractal XL, the features of this board are amazing.
I have a Fractal XL meshify with a load of big fans, airflow is pretty good, of course these serverboards can never get too much airflow, I do try to keep the noise within limits. I put an 1800 rpm Skythe fan in the back, found that the 1500rpm fans leave too much heat around the mem modules.
Would like to get the temps down a bit, was thinking putting a mem-cooler on top of the ddr5, on amazon, unfortunately, there are no reviews on the item.
What do you think, am I just a bit too cautious, if its true the modules can sustain 95 celcius 24/7 ? The memory alone cost me about 2000 euro, so I do want to take care of it and not reduce its lifetime with crazy high temps.

 

[RolloZ170]

I have a similar issue, well maybe the temps are a non issue if 95 celcius is allowed.

lower is better for long life.
DDR5 RDIMM have on DIMM VRM which adds heat to the RDIMM typical clockbuffer heatness.
if we put a "server motherboard" in a PC Case we have to manage the same cooling for the RDIMM section.
server have Airshrouds i.e. not just for prettyness.

 

[Thorvaldt]

lower is better for long life.
DDR5 RDIMM have on DIMM VRM which adds heat to the RDIMM typical clockbuffer heatness.
if we put a "server motherboard" in a PC Case we have to manage the same cooling for the RDIMM section.
server have Airshrouds i.e. not just for prettyness.

Yes I know, Ideally it should have much more cooling, but its not like my build doesn"t have above average airflow, of course its not the same as the airflow a rackserver provides.

 

[Falloutboy]

I can give you maybe a little context on this, albeit with UDIMM not ECC server modules, I have been playing with under clocking memory to tighten timings and over clocking as far as I can and running mixed dims in DDR5 2DPC gaming boards and I can tell you for example that I have a set of ram which is rated for 6000MT , if I try and run it at 6200MT 30-36-26-76 @ 1.4v which is not a ridiculous voltage for a lot of EXPO/XMP DDR5 kits it will fail,
When I say memory tests and I don't mean things like memtest86 because it is possible to have error-ing memory that could still pass in memtest, but it wouldn't with a 24 hour run by Kahu.

If however I place a 40mm fan directly on the memory and just have a little air blow across the heat spreaders it will run at the new speed all the live long day.

While heat can kill some electronics it can make some just get errors, I would not let DDR5 go about 50 degrees, if you do you are asking for it to have problems.

 

[Falloutboy]

What do you think, am I just a bit too cautious, if its true the modules can sustain 95 celcius 24/7 ? The memory alone cost me about 2000 euro, so I do want to take care of it and not reduce its lifetime with crazy high temps.

I have tried that unit in your picture as well as the one from JONSBRO, looks like you may have the room for it, I tried it with a Noctua NH15 with Corsair modules - not enough room for the damned thing to fit. Aliexpress sells those units and might be cheaper - worth checking out.

 

[Thorvaldt]

Thanks for the info, the DDR5 modules already get hot during idling, already ready they reach 50 celcious, probably due to the onboard vrm, like the previous user mentioned. 70 celcious is probably ok, i just don't like two of them reaching 78.
I have a few hundred watts of videocards a really hot SAS controller at the bottom, will place a fan vertically behind these hotheads, so it gets more forced to take the rear exit. I have a feeling, too much of that heat rises to the top. If that doesn't improve i put a fan over the mem modules, but i'm not sure how it fits, the ddr5 are very close to eachother and have a very low profile. The verical bracket i ordered takes another week unfortunately.

 

[compgeek89]

I have a similar issue, well maybe the temps are a non issue if 95 celcius is allowed.
I have a Genoa EPYC 9634 system with 512GB of ddr ecc 4800 mem from Samsung, under heavy load some modules peak at around 78 celcius (in mid summer, temps in room about 30 degrees), while the 6 others stay around 65 degrees. The ones that hit 78, are located below the CPU, the two middle ones, furthers away from the top exhaust ventilators. The CPU hardly gets above 60 degrees nomatter what I throw at it, the big surface really helps.
Mainboard: Asrock GENOAD8X-2T/BCM, which is a humongous board that just about fits the Fractal XL, the features of this board are amazing.
I have a Fractal XL meshify with a load of big fans, airflow is pretty good, of course these serverboards can never get too much airflow, I do try to keep the noise within limits. I put an 1800 rpm Skythe fan in the back, found that the 1500rpm fans leave too much heat around the mem modules.
Would like to get the temps down a bit, was thinking putting a mem-cooler on top of the ddr5, on amazon, unfortunately, there are no reviews on the item.
What do you think, am I just a bit too cautious, if its true the modules can sustain 95 celcius 24/7 ? The memory alone cost me about 2000 euro, so I do want to take care of it and not reduce its lifetime with crazy high temps.

What I ended up doing was putting a Noctua 80mm over each set of RDIMMs like so:

Somewhat makeshift, but RAM temps are 20C lower.

I tried using one of those RAM cooler brackets like you suggest but they just don't fit.

 

[compgeek89]

Ideally you'd have a nice, custom acrylic bracket like this, but, alas, I don't have access to the tools or service for that for a one-off.

 

[drdepasquale]

This worries me especially since I've never experienced any temperature problems with DDR4 RDIMMs. I'm not sure why manufacturers made this design change. Hopefully it is reverted for DDR6.

 

[Thorvaldt]

What I ended up doing was putting a Noctua 80mm over each set of RDIMMs like so:
View attachment 37849
Somewhat makeshift, but RAM temps are 20C lower.

I tried using one of those RAM cooler brackets like you suggest but they just don't fit.

Yes that defenitely would be some kind of solution, I will first try and cool my videocards and my SAS controller a bit more, by placing a fan on a vertical mount behind those cards, blowing to the back end of the case. If that doesn"t suffice I will place a smaller fan behind the lower DDR modules also blowing towards the back. Problem is how I will mount that smaller fan. Thanks for the info that these Amazon brackets won't fit, thats what I thought when I looked at them. Its a bit unfortunate these DDR5 modules are so close to eachother, otherwise I could have wrapped them in some kind of heatsink.

 

[Thorvaldt]

This worries me especially since I've never experienced any temperature problems with DDR4 RDIMMs. I'm not sure why manufacturers made this design change. Hopefully it is reverted for DDR6.

I wonder how long this can continue, these higher frequencies, DDR5 is already getting rediculously hot, even when not overclocking. These NVME modules for PCI5, have a similar problem, I deliberately bought pci4 nvme drives, just because I don't want the extra heat generated by pcie5 drives, let alone future pcie6 drives... the heat produced is gettting a bit crazy with these higher frequencies. Although I am amazed how cool my processor remains, even under heavy load, was expecting much higher temps, looked at a water cooling solution, but 450 euro is just outrageous.

 

[Thorvaldt]

This would be a great product to cool DDR5

, Der Bauer looked at it 2 years ago, seems that the Thermaltake Ramorb is from 2008 and only available in the US. Would love to get my hands on two of them.

 

[drdepasquale]

I wonder how long this can continue, these higher frequencies, DDR5 is already getting rediculously hot, even when not overclocking. These NVME modules for PCI5, have a similar problem, I deliberately bought pci4 nvme drives, just because I don't want the extra heat generated by pcie5 drives, let alone future pcie6 drives... the heat produced is gettting a bit crazy with these higher frequencies. Although I am amazed how cool my processor remains, even under heavy load, was expecting much higher temps, looked at a water cooling solution, but 450 euro is just outrageous.

Agreed. DDR5 and PCIe 5.0 are not matured technologies. Even when DDR4 memory first came out, it didn't have anywhere near the number of stability and temperature problems DDR5 has.

 

[Thorvaldt]

I added two additional fans behind my hot cards, the SAS controller got particularly hot, now more air is flowing along the cards and forced out via the backplane (not via the main fan at the back). The max DDR5 temperatures under heavy load dropped by about 15 degrees celcius(all 84 cores 100% loaded), see screenshots. If I ever add more videocards, I will stack another big fan onto the current big one (Glued the smaller fan ontop of the big fan). Mem A,B,C and E are the modules below the Epyc, you can see the middle ones are hotter, the others are located above the Epyc. Does anyone know what FSC_index would mean, what temperature is being measured ??