Big fan troubles with Supermicro X10SL7-F

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Vocalpoint

New Member
Mar 25, 2016
27
1
3
Calgary, AB
Good morning,

Last week I took delivery of another Supermicro X10SL7-F for my home LAN - this for use as a VM/Deployment server. Since this is my second go around with this board - I am familiar with it's quirks - especially the bloody fans roaring up and down every 5 seconds after the board is powered up and fans attached.

I am also very familiar with adjusting these fans via IPMIUTIL and when I build the box last week - I was able to correctly reset the thresholds and get the fans humming along at a nice quiet pace. The box behaved nicely for 2 or three days last week.

But heading into this weekend - we were going out of town so I powered all the servers off. When we got back yesterday - I power everything back up. My main server (built in April 2016 with a Supermicro X10SL7-F) came back on line quietly as it should. The new build however - came back up with fans wailing away at full blast.

So I pulled the box out of the rack and spent about 2 hours messing around with several versions of IPMIUTIL (v 3.02) and thought I had the fans settled down again. I put the box back in the rack, powered it up and all was well.

Then I wake up and come into my home office this morning with a cup of Joe and see the fans blasting away again. Logs indicate a sudden shift in the CPU fan at around 1:20am to Above Critical - which I do not understand.

What is my next move to get this thing under control? I cannot have the fans wailing away at 1500RPM for months on end.

I am using the stock Intel PWM fan - plugged into FAN1
I have Noctua NF-A14-PWM in FAN 2 and FAN 3
and a Noctua NF-S12A-PWM plugged into FAN4

I think I may need to reset the CPU fan via IPMIUTIL but have no idea what the threshold values should be?

I uploaded a screencap of the fan readings via the IPMI web interface - and they are seriously messed up. (see attachment)

Appreciate any tips or tricks from the field.

Cheers!

VP
 

Attachments

Last edited:

Vocalpoint

New Member
Mar 25, 2016
27
1
3
Calgary, AB
Update

I spent another few hours this morning analyzing fan sensor output from IPMIUTIL - I first extracted the key settings from my first server I built with the x10SL7-F I back in April 2016.

Using this command

ipmiutil sensor -N [SERVER IP} -U [UserName] [Password] -Y -c -g fan -t

I was able to see all the different thresholds for all fans. I took that data and replicated those settings on the new server. Did a quick reboot and it seems to have done the trick. I think the problem was with the CPU fan settings - as soon as I set SERVER 2 to have the same thresholds as SERVER 1 - everything has now settled down - for now :)

Different question now. In these servers - everything is pretty much identical. Same motherboard, same case, same case fans and same location - the two boxes sit beside each other in our server rack. Logically the same ambient temperature etc. They both run WIndows Server 2012R2 and both do nothing more that basic files serving.

The only things that are not identical is the IPMI firmware and the CPU

SERVER 1 is on Firmware v3.27 (redfish 1.0.0) while SERVER 2 is up to date @ firmware 3.45 (redfish 1.01)

SERVER 1 is running a Intel Xeon E3-1220 v3 (LGA 1150) while SERVER2 is running an Intel Xeon E3-1231 v3 (LGA 1150)

The CPU fan on SERVER 1 is at a constant 1100 RPM with case fans each running around 500RPM while SERVER 2 has it's CPU fan running constant at 1000RPM and the cases fans each running around 400RPM.

Given the RPM difference - I immediately noticed that SERVER2's CPU temp (38 degrees C) is 10 degrees higher than SERVER1 (28 degrees C.)

I also noticed that the LowNR and LowCT settings on SERVER1 are different than those on SERVER2 - even when using the identical settings (from Server1) to program SERVER 2's fan thresholds.

On SERVER 1 LowNR = 300 and LowCT = 500
On SERVER 2 LowNR = 500 and LowCT = 600

Would it be the firmware causing a difference or a more energy efficient CPU accounting for the difference?

While I do not believe we have a thermal issue on the CPU for SERVER2 - this temp difference is a slight concern - especially when the CPU fan is spinning slower on SERVER2 - which could very well be the reason for the 10 degree rise in temp?

Thoughts?

VP
 

FlashEngineer

Member
Jan 27, 2016
96
3
8
You can set fan speed using ipmitool from shell prompt in linux as long as your board has IPMI enabled etc.

/usr/bin/ipmitool -H 10.0.0.50 -U admin -P admin raw 0x30 0x70 0x66 0x01 0x00 0x29

The last hex value is fan speed, it's based from 0x00 to 0x64 which 64 is 100%.

Set the fan mode to full speed, then you can adjust with command line. That's how I set all my supermicro fan speeds.
 

Vocalpoint

New Member
Mar 25, 2016
27
1
3
Calgary, AB
Set the fan mode to full speed, then you can adjust with command line. That's how I set all my supermicro fan speeds.
Does the actual Fan mode have anything to do with setting the actual thresholds?

Also - I am using IPMIUTIL (For windows workstation - not Linux) but the intent is the same:

More interesting is another fan "event" seen last night after simply restarting a service on the machine.

Event logs within Windows show the service was successfully restarted @ 8:24pm and then all of a sudden - the fans started wailing away at full blast.

So I check the Event log for the x10SL7-F via IPMI - and sure enough - it recorded and "UPPER CRITICAL Assertion" starting with FAN3 - which is my front intake fan.

My case fan settings as of last night looked like this:

FAN2 (Snum 42) = 140mm Top Exhaust

ipmiutil sensor -N 192.168.1.6 -U ADMIN -P ADMIN -n 42 -l 300 -h 800

FAN3 (Snum 43) = 140mm Front Intake

ipmiutil sensor -N 192.168.1.6 -U ADMIN -P ADMIN -n 43 -l 300 -h 800

FAN4 (Snum 44) = 120mm Rear Exhaust

ipmiutil sensor -N 192.168.1.6 -U ADMIN -P ADMIN -n 44 -l 300 -h 800


I always thought the fans only went "full" when a lower threshold was encountered (making the motherboard think "Overheating is about to occur - I better put the fans on high alert!" - but I have never seen a Supermicro board do this - placing ALL fans on full becuase an "upper" threshold is suddenly exceeded.

I am also weirded out by how a fan can go from 350-ish RPM to over 800RPM (simply because a service was restarted) but I guess it's possible.

Has anyone encountered what I am seeing - a "high" fan threshold being overshot and fans going nuts?

I have since increased the case fans threshold to 1000 to see if it makes a difference - but would like to know what you guys are setting your high threshold to. Maybe I am way too low here.

Again - Noctua NF-A14-PWM fans are on FAN 2 and FAN 3 and a Noctua NF-S12A-PWM plugged into FAN4

Appreciate any settings or tweaks you can share.

Cheers!

VP
 

Media_Server

New Member
Feb 28, 2017
2
0
1
124
I recently went through the SuperMicro fan speed revving up and down issue that SuperMicro MOBOs seem to be famous for.
I have the X9SRi-F.

  1. Download the Documentation for your MOBO from Supermicro's Website on Setting IPMI
  2. Make sure IPMI is active on your MOBO according to Supermicro's Instructions (this isn't really complicated... just some settings in BIOS).
  3. Hook up a HDD, Keyboard, Mouse & Monitor and install a Linux distro. (I believe I used Fedora but it may have been Mint).
  4. Boot into Linux on your server with and get to work at the command line.
**If you try to do this remotely using the IPMI network connection then this guide won't be as helpful, you'll have to do some adapting. Much simpler to just hook up the monitor and do it directly.


FIRST TO SET FAN THRESHOLDS FOR SUPERMICRO MOBO TO TURN OFF ANNOYING RAMPING UP AND DOWN OF FANS:


su
modprobe ipmi_msghandler
modprobe ipmi_devintf
modprobe ipmi_si
service ipmi restart
ipmitool sensor
the last command will give you a readout of all sensors, including the fan sensors.
You will see the current fan speeds are for the fans you have connected along with their sensor designator.
This is where I got tripped up for a day or so. Different SuperMicro motherboards will list the fan speed sensors
differently.

Some will be:
Fan1
Fan2
Fan3
...
...
etc.

Mine listed them as:
Fan 1
Fan 2
Fan 3
...
...
etc.

Because My MOBO readout included a space in the fan sensor designators I had to alter the code from what I found
on the net to accomodate that space. Solution (probably bonehead-simple to anyone with experience but not so much
to an utter novice like me was to put a single quote ---> ' ...around my sensor numbers in the following code:

ipmitool sensor thresh 'FAN 4' lower 200 225 300
if your ipmitool sensor readout didn't have a space between the word FAN & the number 1,2,3,4,etc... then don't include the quote
and your code would look like this instead:

ipmitool sensor thresh FAN4 lower 200 225 300
this assuming that your Fan was located at the #4 fan position.

So let's paste it all into a single box... all comands needed to change the FAN thresholds.


su
modprobe ipmi_msghandler
modprobe ipmi_devintf
modprobe ipmi_si
service ipmi restart
ipmitool sensor
ipmitool sensor thresh FAN2 lower 200 225 300
ipmitool sensor thresh FAN3 lower 200 225 300
ipmitool sensor thresh FAN4 lower 200 225 300
alternatively

su
modprobe ipmi_msghandler
modprobe ipmi_devintf
modprobe ipmi_si
service ipmi restart
ipmitool sensor
ipmitool sensor thresh 'FAN 2' lower 200 225 300
ipmitool sensor thresh 'FAN 3' lower 200 225 300
ipmitool sensor thresh 'FAN 4' lower 200 225 300


That's it... almost.

When I did this I encountered a new problem, my fans were all now running at maximum speed 100% of the time.
SuperMicro MOBOs have a Fan Speed Setting that needs to be changed as well.



-------------------

THEN TO SET FAN SPEED SETTING FROM COMMAND LINE:



1st, check current fan speed setting:
ipmitool raw 0x30 0x45 0x00
This will return the current fan speed setting which will be either 00, 01, 03, or 04 where:

00 = Normal BMC control of both fan zones, with CPU zone based on CPU temp (target speed 50%) and Peripheral zone based on PCH temp (target speed 50%)
01 = Full All fans running at 100%
02 = Optimal BMC control of the CPU zone (target speed 30%), with Peripheral zone fixed at low speed (fixed ~30%)
04 = Heavy IO BMC control of CPU zone (target speed 50%), Peripheral zone fixed at 75%



2nd, actually set the global fan speed setting:

ipmitool raw 0x30 0x45 0x01 0x0# (where # = 0. for Normal, # = 1 for Full, # = 2 for Optimal)


Example of command that I used (that worked):
ipmitool raw 0x30 0x45 0x01 0x00
 

Media_Server

New Member
Feb 28, 2017
2
0
1
124
I always thought the fans only went "full" when a lower threshold was encountered (making the motherboard think "Overheating is about to occur - I better put the fans on high alert!" - but I have never seen a Supermicro board do this - placing ALL fans on full becuase an "upper" threshold is suddenly exceeded.

I am also weirded out by how a fan can go from 350-ish RPM to over 800RPM (simply because a service was restarted) but I guess it's possible.

Has anyone encountered what I am seeing - a "high" fan threshold being overshot and fans going nuts?

I have since increased the case fans threshold to 1000 to see if it makes a difference - but would like to know what you guys are setting your high threshold to. Maybe I am way too low here.
okay, I actually went and read through your post and I see that you are experiencing the exact same thing I did when I lowered all my thresholds.

The problem isn't with the thresholds, it's with a more global fan speed setting on the SuperMicro MOBOs.

They're two separate things. I tried what you suggest at 1st, raising the upper threshold and it did not help me. Keeping the upper threshold very low worked just fine when I adjusted the more global fan speed setting. See the 2nd half of my long post above and you'll see what I'm referring to.

I have no idea how this will translate into the windows IPMIUTIL.
If what I've written doesn't work directly with the windows IPMIUTIL you might consider the Linux route... just find a small disk and install Linux on that on your server for the 10 minutes it takes to do the IPMI changes. Then swap it out for whatever you want to run after that. Fedora is a breeze to install and work with. Anyway, good luck.