Zeus V2 : U-NAS NSC-810A | X10SL7-F | E3-1265 V3

[dlinsley]

Now I am using supermicro sas 8643 to sata breakout cables for the backplane and a different short sata cable for the OS drive.

Hi, which length breakout cable is needed? I'm thinking the 20cm are likely fine, but I'd like to order the cables from Supermicro while my case is in transit.

Cheers,
David.

 

[Juan Manuel Palacios]

Hey everyone!

Hopefully this will be the best thread to publish this follow-up question, but if not please do point me at one.

So, as you know, my build is the U-NAS 810A box (stock fans, all three), with a Supermicro X11SSH-LN4F board and an Intel Xeon E3-1240 v6 processor (stock cooler & paste), and during the last weeks I've been burning in the different components. This weekend it was the CPU's turn, for which I attempted an mprime stress test run that was unfortunately very short-lived.

In a matter of seconds the CPU temperature shot up to 100C, and the case/board very fortunately started screaming very loudly about it (because otherwise I wouldn't have caught it). Then I got all the fans blowing at full speed and turned on the room AC, and managed to get the test running for a few minutes at 91-92C stable, but canceled it again as I don't feel anywhere near comfortable running the CPU at that temperature for any extended periods of time.

As you all know, cable management in this box is not that simple, and the area between the back plate and the back fans is a little crowded with cables. What would you suggest to improve heat management? I've read about people running mprime's stress test at around 60C CPU temperature, which seems a lot more reasonable than anything approaching 100C. Would the Noctua coolers that we already discussed have a sizable impact over the stock Intel cooler? Should/may I add another cooler on top of that as well (the X11SSH-LN4F board does have an extra fan connector I could use)? And, if so, how/where could I hook it up?

Thanks again for any pointers!

 

[K D]

Hey everyone!

Hopefully this will be the best thread to publish this follow-up question, but if not please do point me at one.

So, as you know, my build is the U-NAS 810A box (stock fans, all three), with a Supermicro X11SSH-LN4F board and an Intel Xeon E3-1240 v6 processor (stock cooler & paste), and during the last weeks I've been burning in the different components. This weekend it was the CPU's turn, for which I attempted an mprime stress test run that was unfortunately very short-lived.

In a matter of seconds the CPU temperature shot up to 100C, and the case/board very fortunately started screaming very loudly about it (because otherwise I wouldn't have caught it). Then I got all the fans blowing at full speed and turned on the room AC, and managed to get the test running for a few minutes at 91-92C stable, but canceled it again as I don't feel anywhere near comfortable running the CPU at that temperature for any extended periods of time.

As you all know, cable management in this box is not that simple, and the area between the back plate and the back fans is a little crowded with cables. What would you suggest to improve heat management? I've read about people running mprime's stress test at around 60C CPU temperature, which seems a lot more reasonable than anything approaching 100C. Would the Noctua coolers that we already discussed have a sizable impact over the stock Intel cooler? Should/may I add another cooler on top of that as well (the X11SSH-LN4F board does have an extra fan connector I could use)? And, if so, how/where could I hook it up?

Thanks again for any pointers!

See my pictures in the OP. I routed the cables through the space between the fans and hdd backplane.

Can you post pictures of your build?

 

[Juan Manuel Palacios]

Hi @K D!

I saw your pictures when assembling my build, they were indeed an important reference. As a result my cable routing is similar to yours (but probably not as neat), but also because I didn't see much of an alternative when trying to keep the cables grouped together, off of the motherboard, etc.

Things got a little worse when I switched from the U-NAS SATA cables, which are nicely thin but otherwise seem kinda low quality, to the Supermicro ones, which seem a lot more robust but are also definitely a lot more bulky. That ate up quite a bit of the space at the rear, unfortunately. I'll snap some pics and upload them when I get home.

Still, you think that potential poor cable handling at the rear could be to blame for the CPU shooting up to 100C so fast, when it's got a fan literally sitting right on top of it? Could it be that the hot air that's being generated is not being exhausted out of the sealed case too efficiently, and it's being blown right back on to the CPU? You do feel a nice air current coming out of the back fans when you place your hand behind them...

Could it be that the thermal paste that the stock Xeon cooler comes with may not have been enough? And/or that the cooler is a little too anemic?

OK, OK, too many hypotheticals, I'll post the pics ASAP!

 

[K D]

I assumed that the cables were being routed through the top area and were obstructing the exhaust fan. . As you have mentioned, try cleaning off the existing thermal paste and re seat the cooler. Let it run at idle for a few hours for the thermal paste to settle and then try the stress test again.

Also check whether the CPU fans and exhaust fans are connected and running at full speed while on load.

 

[Juan Manuel Palacios]

OK, good tips! But to do that I'm gonna have to get new paste, as I don't have any right now (recommendations for my chip?), and learn how to control the CPU fan speed, as the Supermicro IPMI console only allows me to set the speed of the entire collection of fans at once to different levels. But if that includes the CPU fan, then I'm all set on that front; I'll open up the case to confirm. On a side note, I'd be great if the fan speed control could be dynamic and respond to heat thresholds!

But back to the case and fans, you did say CPU "fans"... by that are you also referring to the one on the side of the case, or something else? I have the two at the back that came with the case itself, which I'm guessing are the ones you're calling "exhaust fans", the CPU fan from Intel right on top of the heatsink, and this one to the side (left if you look at the case from the front) also provided by U-NAS. My board (X11SSH-LN4F) has one extra fan plug that's currently unused right now, to the back and close to the NICs, but I wouldn't know where/how I could hook up a fan to use it.

I'll take the pics and post them tonight.

 

[K D]

I use either the Noctua NH-T1 or Arctic Silver 5 thermal paste.

I meant CPU "Fan" (The one on the cooler) and the Exhaust "Fan" (The 70mm exhaust fan in the top part of the case). Connect both these to FAN1-FAN4 headers on the motherboard.

Connect the 2 120mm fans behind the HDD cage to FANA and FANB headers on the motherboard.

Supermicro has 2 zones (Zone 1 for FAN1-4 and Zone 2 for FAN A and FAN B).

 

[Juan Manuel Palacios]

Hi @K D!

Which one is Fan B? I only have Fan 1, 2, 3, 4, and A on the X11SSH-LN4F. Here's an image with the diagram of a very simular board, you'll see header 4 to the top (back) right and next to COM1, and 1, 2, 3, and A to the bottom (front) and moving towards the left right below the RAM slots.

This post tells me to connect the CPU fan to one of the numbered headers, but the Supermicro reference guide seems to suggest I should connect it to header A, which is where I currently have it. Are those two indeed in contradiction? Or am I reading something wrong?

My 70mm fan is currently connected to header 3, and the two back 120mm fans to headers 1 and 2, leaving header 4 currently unused.

Also... and hopefully this wont be a dumb question, but in which direction should CPU fan air flow? Towards the CPU? Or away from it? Logic tells me it should be the latter (even though I'm currently doing the former), to pull the hot air away from the CPU and hopefully exhaust it out of the case, but I just realized there's hardly consensus on this on the internet.

In any case, here are my pictures, as best as I could take them, on Google Drive because I could't upload them to this forum ("an error occurred" every time I tried):

NAS Build - Google Drive

 

[K D]

I just checked the manual and it has only FANA

https://www.supermicro.com/manuals/motherboard/C236/MNL-1777.pdf

Go with the recommendation in @PigLover s guide that you linked. I have followed those instructions in x9, x10 and x11 boards without issue.

This is what I had done
CPU fan - FAN2
70mm Fan - FAN3
2 HDD Fans - FANA - Use a Fan splitter cable (There was one that came with the Chassis)

CPU AirFlow direction - I usually direct it towards the CPU as that's how most of the coolers that I have seen had their default setup.

 

[EluRex]

You can check my build @ PVE Cluster using UNAS 810A we are using the same motherboard and same cpu (mine is E3-1245v6) and my cpu temp never exceeds 50c under full load.

I did rearrange all the backplane's 4 pin atx cable straight from the psu instead using the default cabling.

Make sure you use a low profile cpu heatsink to give it a better air flow from the cpu fan.

Hey everyone!

Hopefully this will be the best thread to publish this follow-up question, but if not please do point me at one.

So, as you know, my build is the U-NAS 810A box (stock fans, all three), with a Supermicro X11SSH-LN4F board and an Intel Xeon E3-1240 v6 processor (stock cooler & paste), and during the last weeks I've been burning in the different components. This weekend it was the CPU's turn, for which I attempted an mprime stress test run that was unfortunately very short-lived.

In a matter of seconds the CPU temperature shot up to 100C, and the case/board very fortunately started screaming very loudly about it (because otherwise I wouldn't have caught it). Then I got all the fans blowing at full speed and turned on the room AC, and managed to get the test running for a few minutes at 91-92C stable, but canceled it again as I don't feel anywhere near comfortable running the CPU at that temperature for any extended periods of time.

As you all know, cable management in this box is not that simple, and the area between the back plate and the back fans is a little crowded with cables. What would you suggest to improve heat management? I've read about people running mprime's stress test at around 60C CPU temperature, which seems a lot more reasonable than anything approaching 100C. Would the Noctua coolers that we already discussed have a sizable impact over the stock Intel cooler? Should/may I add another cooler on top of that as well (the X11SSH-LN4F board does have an extra fan connector I could use)? And, if so, how/where could I hook it up?

Thanks again for any pointers!

 

[Juan Manuel Palacios]

Hey @K D, @EluRex, everyone!

Thanks for the help so far and for the useful tips! Sorry for taking so long to reply, I had to order Arctic Silver 5 (and the cleaner kit), and then work kept me from working on my box for a while.

@K D, I rearranged the fan cables as you suggested, i.e. CPU fan on header 2 & HDD fans through the splitter and on to header A, and then I proceeded to clean up my CPU & heatsink and reapply the thermal paste. I'm running mprime's stress test right now, and though my system's doing a little better, it's still up to 88C (even with the case open and all of the fans at full speed), which I still don't feel to comfortable with. On the plus side, it does seem to be stable at that temperature for a good number of minutes now, only fluctuating +/- 1C every now and then.

@EluRex I just checked your pics, thanks for the pointer! What heatsink are you using? What were the cables that you rearranged, the HDD power ones? How? It's not too clear from the pics.

All in all, any other tips anyone could give me to get this CPU temp down? After around 30 mins into typing this reply, it's still at 88C running the stress test, but I don't think I'll let it run for too much longer like that (even though all tests are passing fine). Intel's CPU installation tutorial talks about using their BXTS13X liquid cooling solution, but that seems to require space for 120mm side-mounted fan, and the U-NAS 810 only has space for a 70mm, if I'm not mistaken. Any similar solution you would suggest that would fit this case? Or would this BXTS13X work with the U-NAS-provided 70mm fan (I understand the the fan in the BXTS13X can be replaced aftermarket, but I'm not sure if a smaller fan would be good for the job)?

Thanks again for all the help!

 

[K D]

Is this with the fans running at 100% (Fan mode=Full in IPMI). With the case open and the CPU fan running at full, you should not be seeing any issues. Could be a problem with the CPU or motherboard.

This case does not support a watercooling solution. The cooler that @EluRex has looks like a Gelid Slim Silence I-Plus cooler.

 

[Juan Manuel Palacios]

Is this with the fans running at 100% (Fan mode=Full in IPMI). With the case open and the CPU fan running at full, you should not be seeing any issues. Could be a problem with the CPU or motherboard.

This case does not support a watercooling solution. The cooler that @EluRex has looks like a Gelid Slim Silence I-Plus cooler.

Hi @K D! Yes, the 88C +/- 1C is indeed with the fans at full speed; I edited my message a little while ago to clarify this.

When you say I shouldn't been seeing any issues with the case open and the fans at full, you mean that it's OK to run the CPU at 88C (obviously only while the stress test runs)? Or that I should, on the contrary, not be experiencing these temperatures, and that there's in fact something wrong?

And about the Intel liquid cooling solution, I wont be able to install any of them in this case, other than the size of the fan? Why's that? (I emphasize, I'm still a n00b at building hardware!).

The Gelid Slim Silence I-Plus cooler is only $35 on newegg, I could easily get it. Do you think it's gonna have sizable impact on my temps?

 

[K D]

Hi @K D! Yes, the 88C +/- 1C is indeed with the fans at full speed; I edited my message a little while ago to clarify this.

When you say I shouldn't been seeing any issues with the case open and the fans at full, you mean that it's OK to run the CPU at 88C (obviously only while the stress test runs)? Or that I should, on the contrary, not be experiencing these temperatures, and that there's in fact something wrong?

And about the Intel liquid cooling solution, I wont be able to install any of them in this case, other than the size of the fan? Why's that? (I emphasize, I'm still a n00b at building hardware!).

The Gelid Slim Silence I-Plus cooler is only $35 on newegg, I could easily get it. Do you think it's gonna have sizable impact on my temps?

From what I counld find after some searching, the max safe temp for E3-1245 v5 is around 75degrees. Honestly I don't know if 88degrees at load is a safe temperature but I am seeing lower temperatures with a bigger CPU. I got ~68deg with the Noctua NHL9-i and E3-1275 v6 on a test bench which would be the equivalent of leaving the top open for this case.

Have you tried the CPU in another motherboard or this motherboard with another CPU?

This case is too small to install any liquid cooling solutions that I know of. May be a custom one can be possible.

 

[Juan Manuel Palacios]

From what I counld find after some searching, the max safe temp for E3-1245 v5 is around 75degrees. Honestly I don't know if 88degrees at load is a safe temperature but I am seeing lower temperatures with a bigger CPU. I got ~68deg with the Noctua NHL9-i and E3-1275 v6 on a test bench which would be the equivalent of leaving the top open for this case.

Have you tried the CPU in another motherboard or this motherboard with another CPU?

This case is too small to install any liquid cooling solutions that I know of. May be a custom one can be possible.

Thanks for the info. Does that bench test push the CPU as hard as mprime's stress test does? 68C definitely seems like a very comfortable number to have when running under this amount of activity.

I haven't tried any other combination of CPU & board & case because I currently don't have any other than my current parts at hand, unfortunately.

So about the liquid cooling solution, you definitely think I'm gonna be wasting my money if I buy Intel's for this case? (sorry to sound a bit repetitive, I feel I'm running out of options!).

 

[Juan Manuel Palacios]

Oh, and also, what was it that you meant by the "you should not be seeing any issues." comment?

When you say I shouldn't been seeing any issues with the case open and the fans at full, you mean that it's OK to run the CPU at 88C (obviously only while the stress test runs)? Or that I should, on the contrary, not be experiencing these temperatures, and that there's in fact something wrong?

 

[EluRex]

....
@EluRex I just checked your pics, thanks for the pointer! What heatsink are you using? What were the cables that you rearranged, the HDD power ones? How? It's not too clear from the pics.
....

I am using Gelid Slim Silence I-Plus cooler, it has a lot of head room spacing for the cpu fan.

Before ATX power cable rearrange. the back panel is using a 4pin splitter connect to the psu

After rearrangement. The ATX power cable is direct from the PSU itself.

The splitter cable removed

 

[Juan Manuel Palacios]

Hi @EluRex!

Thanks for those pics, very useful! Some more questions, if I may:

1) Did you find the Gelid Slim Silence I-Plus cooler to have an important impact on CPU temperatures? As stated, I'm using the stock cooler provided by Intel with the Xeon E3-1240 V6, and after re-applying Arctic Silver 5 paste strictly following Arctic Silver's guidelines for this CPU (tinting of CPU & heatsink + line method), my CPU is still doing 87C under mprime's stress test. I've improved 5C since I first tried, but I'm still uncomfortable with those temps, and I'd be willing to buy a new cooler if it's really gonna have an important impact.

2) That cable rearrangement definitely looks interesting! Other than my CPU, my system doesn't seem to be having any heat issues, with the drives operating at very cool mid-30C even under badblocks stress test. But, still, my back panel is definitely a little overcrowded, and could use some improvements. Did you have to buy any extra (long?) cables to get rid of that 4pin splitter? I'm using the SeaSonic SS-350M1U 350W PSU, not sure if that already brings with it all that I need...? (not currently in front of it).

Thanks!

 

[K D]

I think you should experment with other cpu coolers first. The cabling between the hard drive and fans will not affect your CPU temperatures. Try the Noctua NH-L9i or the Gelid Slim Silence suggested by @EluRex

@EluRex : Unrelated Question : In your build log, i see you had experimented with reversing the Fans in the HDD cage. Did that make a big difference? I am using NOctua iPPC 2000s and have to run them at full speed to maintain 8TB Reds at ~34C. Unfortunately I have already "deployed" the server and it's in the middle of a copying several terabytes over a single GBe Link and I probably wont be able to touch it for several days. If reversing the fans makes a difference, It's worth bringing it down and updating it. Else, I'll just leave it as is.

 

[EluRex]

Hi @EluRex!

Thanks for those pics, very useful! Some more questions, if I may:

1) Did you find the Gelid Slim Silence I-Plus cooler to have an important impact on CPU temperatures? As stated, I'm using the stock cooler provided by Intel with the Xeon E3-1240 V6, and after re-applying Arctic Silver 5 paste strictly following Arctic Silver's guidelines for this CPU (tinting of CPU & heatsink + line method), my CPU is still doing 87C under mprime's stress test. I've improved 5C since I first tried, but I'm still uncomfortable with those temps, and I'd be willing to buy a new cooler if it's really gonna have an important impact.

2) That cable rearrangement definitely looks interesting! Other than my CPU, my system doesn't seem to be having any heat issues, with the drives operating at very cool mid-30C even under badblocks stress test. But, still, my back panel is definitely a little overcrowded, and could use some improvements. Did you have to buy any extra (long?) cables to get rid of that 4pin splitter? I'm using the SeaSonic SS-350M1U 350W PSU, not sure if that already brings with it all that I need...? (not currently in front of it).

Thanks!

Juan,

1) I know the stock cpu cooler fits but I never used it due to it does not have enough spacing for cpu fan to operate at efficient to generate air flow. Like KD suggested its best for you to explore other cpu cooler. Gelid is just quiet and my favor is still Noctua but $$$

2) yes I did use 55cm long atx power cables but my psu is same as yours.

.....

@EluRex : Unrelated Question : In your build log, i see you had experimented with reversing the Fans in the HDD cage. Did that make a big difference? I am using NOctua iPPC 2000s and have to run them at full speed to maintain 8TB Reds at ~34C. Unfortunately I have already "deployed" the server and it's in the middle of a copying several terabytes over a single GBe Link and I probably wont be able to touch it for several days. If reversing the fans makes a difference, It's worth bringing it down and updating it. Else, I'll just leave it as is.

Reversing the fan didn't work and actually perform worse... so I switched back. Your HDD temp is OK , I am using 7200 rpm HDD as well. There is not much we can do except increase air circulation like below pics using usb fans (blowing outwards)