Epyc 3251 thermal paste replacement

[altano]

I'm having a lot of trouble passively cooling my Epyc 3251 CPU on the M11SDV-8C-LN4F in a 2U case. I'm even ducting a fan to the heatsink and it's not enough: the CPU hits 90 degrees under full load and I have to abort the test. I thought I would try replacing the thermal paste on the CPU (since SuperMicro isn't known for their excellent quality thermal paste) before giving up and moving everything into a 1U case or finding some active cooling (e.g. the 8C+ version of the board).

When I pulled back the CPU heatsink I found a very small die with some paste on it:

I was surprised by the size of the area that had paste on it but set to work wiping off the equivalent area on the heatsink when I realized that the heatsink had some kind of thermal paste or thermal pad on the entire area, not just where the die had thermal paste on it:

Now I'm in a little over my head and not sure what to do. SuperMicro's paste/pad is staying on the heatsink and not getting all over the CPU so one option is to leave that part of the paste alone (although I've already smudged it) and just replace the paste I already wiped off, the part that goes over the die above, which looks like more traditional thermal paste.

Anyone here less of an idiot than me know what the actual right thing to do here is?

 

[EffrafaxOfWug]

I have to confess I'm not sure what the conundrum is here. I'm not sure what you mean when you say "SuperMicro's paste/pad is staying on the heatsink and not getting all over the CPU" - the current paste is clearly all over the CPU. Don't you just want to de-gunk the heatsink and CPU (using surgical spirit/isopropyl alcohol and a cotton bud with a microfibre wipe-down is my preferred method; don't use acetone!) and apply a new blob of $thermal_compound_of_choice?

Or is it that you're worried about excessive clearance between the CPU and the heatsink itself?

 

[altano]

I'm not sure what you mean when you say "SuperMicro's paste/pad is staying on the heatsink and not getting all over the CPU" - the current paste is clearly all over the CPU. Don't you just want to de-gunk the heatsink and CPU (using surgical spirit/isopropyl alcohol and a cotton bud with a microfibre wipe-down is my preferred method; don't use acetone!) and apply a new blob of $thermal_compound_of_choice?

Or is it that you're worried about excessive clearance between the CPU and the heatsink itself?

If I replace this thermal paste:

with traditional thermal paste, it will get all over this area of the CPU:

Which currently has NO thermal paste on it because the current thermal paste has stayed completely on the heatsink.

So do I just leave that area alone and try to replace the paste exactly on the die or can I do better here?

 

[Patrick]

The die is the only part that needs thermal paste. The rest of the PCB is primarily to distribute pins to the BGA underneath.

I am surprised you are having issues. We have 2x 3251's, 1x 3201, 1x 3151, and 1x 3101 all being cooled without problems using the stock heatsink.

 

[altano]

Thanks for the info, Patrick.

We have 2x 3251's, 1x 3201, 1x 3151, and 1x 3101 all being cooled without problems using the stock heatsink.

1U case or 2U? And what sort of fans are you running? I'm running a 2U w/ 3x Noctua NF-A8 PWM fans which max at 2100 RPM.

 

[altano]

Like so:

 

[Patrick]

Oh wow! All of them are in 1U cases or they have air ducting directly over the heatsink.

Especially with those Noctua fans you are losing all of the air pressure. You need much tighter ducting over the CPU.

 

[altano]

Yeaaaa. My quest to keep my rack volume from spiraling has been a failure. My storage server HD temps shot up from ~30°C to ~50°C when I switched to these fans so I had to switch back to the originals.

Any suggestions for better 2U cases or ducting for this use case? Or higher RPM/pressure fans that can still throttle down to low-RPM on idle?

Or should I just give up and move this into a 1U case?

 

[Patrick]

This is a great use-case for a 3D printer

 

[EffrafaxOfWug]

If low noise is what you're aiming for then 1U should be crossed off straight away; it's impossible to fit in anything much bigger than a 40 or 50mm fan, and those can only move decent amounts of air at high rpm and are very loud in doing so. Just how "quiet" are you looking to get anyway?

From the pic you posted, those are noctua 80mm - they don't make a high static pressure version of these TTBOMK, so as Patrick says you're losing a lot of airflow. There's also a big gap between the fans the the components that need cooling - without ducting the air will take the path of least resistance, and it looks like that path will avoid much of the bits that actually need cooling. In all the rackmount cases I've used the fans are much closer to the motherboard (usually due to there being hot-swap bays, then a fan wall, then the motherboard); if moving the fans closer to the mobo isn't possible, then knocking up some basic ducting (card or plastic sheets and a scalpel much cheaper, faster, more reliable and more flexible than 3D printing IMHO) is likely the simplest solution. For starters, just a single vane that converges the three fans on to the leading edge of the motherboard would likely make a big improvement - that's easily testable with a single rectangle of card from your cornflakes and a few strips of tape.

Re: your original topic; yeah, just clean off the gunk from both surfaces and apply the TIM to just the CPU die - then put the heatsink back in place (but not fastened) to check the surfaces mate correctly.

 

[pricklypunter]

The heatsink had one of those morphing thermal pads on it, half of which has broken away on the CPU die. Clean both the CPU die and the bottom of the heatsink completely with some Isopropyl. For compound, I use Arctic Silver Ceramique, but really, any good paste will do you. It really isn't rocket science, there is very, very little to choose between them in thermal characteristics, despite all the marketing wank that these companies would have you believe. Just don't use the old metallic based stuff and you'll be good to go. Place two or three wee globs onto the CPU die, about 1/3 the width of it in diameter and put your heaksink back on carefully, the contact pressure will level and spread it across the surface of the die as you tighten it down.

As @Patrick says, WOW. That type of heatsink is normally actively cooled with a proper fan duct and high static pressure fans, but for the time being, bend yourself up some cardboard to make a snug fitting shroud, stick a couple of cheap fans on it at either end, they don't need to be big in size, just turning at a decent rate with moderate static pressure. Configure them as one pushing and one pulling. Use some duct tape or something just as strong to hold it together, then place it over the heatsink. No need to actually attach it to anything, it's own weight will hold it in place. It will not be pretty to look at, unless you're handy with crayons, but it will get you out of trouble until you find a permanent solution

 

[MiniKnight]

What I'd do is take another noctua fan and some wire and stick it to the top of that heatsink turning it into an active cooler. These are low TDP parts so you won't have issues cooling it that way.

 

[ttabbal]

What I'd do is take another noctua fan and some wire and stick it to the top of that heatsink turning it into an active cooler. These are low TDP parts so you won't have issues cooling it that way.

Glad it's not just me. I did that with an older Opteron I was using as a firewall. I set a fan on top while testing to keep it cool, didn't take much, just a random case fan. Then I used a fan bracket generator on Thingiverse to 3D print a bracket that fit nicely over the heatsink. The fins were the super thin type that wouldn't hold a screw.

For the OP, do that, or make a duct with high static pressure fans. The Noctuas are great, but they aren't moving enough air and probably aren't high enough pressure. There's a reason server fans are loud. One of the Noctuas set on the heatsink would probably cool it though. Then you just need to make sure you're moving enough air to keep the inside of the case from getting super hot. Venting the PSU heat would help too, but there's only so much you can do sometimes. (Got a better look at the PSU, think you're OK There)

Could be worse, I think my Threadripper raised the room temp about 10C.

 

[Evan]

I have that on a d-1500 board, works fine.

 

[nthu9280]

Glad it's not just me. I did that with an older Opteron I was using as a firewall. I set a fan on top while testing to keep it cool, didn't take much, just a random case fan. Then I used a fan bracket generator on Thingiverse to 3D print a bracket that fit nicely over the heatsink. The fins were the super thin type that wouldn't hold a screw.

You are not alone. I've also put a 60 mm fan on 2U SNK-0048P / PS but was able to use screws. they were just keeping the fan with friction grip. Didn't even have any indents on the fins where the screws were. I've since moved on to Dynatron 2u active heat sink though.

 

[Patrick]

The AMD EPYC 3000 heatsink I think is bigger, but this is at least a good start for the concept
https://www.servethehome.com/near-silent-powerhouse-making-a-quieter-microlab-platform/

 

[altano]

If anyone's following along, I ended up switching to a 1U CSE-504-203B chassis and 2 x FAN-0100L4 fans. Under a StressLinux CPU stress test, the Epyc 3251 temps stabilize at 65°C and the fans at 5000RPM.

(a) I'm actually very happy with the volumes, both at idle and at load. It isn't too loud for my office closet.
(b) There is no universe in which I would have been able to get the same airflow I'm getting now by ducting the Noctuas in my 2U. That was clearly futile. The idea of finding some way to secure a fan to the top of the heatsink probably would have worked though.

 

[altano]

And (c) Supermicro told me they are going to sell the active CPU fan on the 8C+ model separately, but it's not ready yet.

 

[Rolando]

Wow, I’m about to get the same motherboard, I never thought that this cpu didn’t had a heat-spreader. Help me out with something, is just me or the socket size is not that big as the SP4/TR4 of the EPYC 7K series?, if that is the case, I could use a Noctua CPU cooler like the Nh-l12s?, any chance you can share the distance between the bracket mount points? Just to check if I would need to mod the brackets of the cpu cooler?. Also that heatsink looks a lot like the one included with the Xeon D-2100, any chance that supermicro are using the same heatsink and that the cpu brackets are the same? Thanks!

 

[Rolando]

Also, does the heatsink have an SKU? so I can search in the web of supermicro for some reference