P

Supermicro X9/X10/X11 Fan Speed Control

Infantryman

New Member
Mar 28, 2021
1
1
1
For those Windows users who wish to change IPMI server settings remotely, use SMCIPMITool for Windows.

Ensure your Command Prompt utility is pointing at the directory where the SMCIPMITool.exe file is located.

Here is a sample to set CPU fan @ 50%: SMCIPMITool.exe 192.168.0.100 myusername mypassword ipmi raw 30 70 66 01 00 32

Here is a sample to set Case/Chassis fans @ 50%: SMCIPMITool.exe 192.168.0.100 myusername mypassword ipmi raw 30 70 66 01 01 32

Green Value = CPU or Case/Chassis Fans (00 for CPU or 01 for Case/Chassis)

Blue Value = Fan Speed (Change value "32" to your taste. 00 = 0%, 16 = 25%, 32 = 50%, 64 = 100%. Nothing stop you from using other values inside the 00 - 64 scope. If you want it @ 75%, then use value 48.)

If you get "00" after issuing the command, your settings has been accepted by the server.

Hope this helps!
 
Last edited:
  • Like
Reactions: dawsonkm

kodi

New Member
Aug 5, 2020
15
0
1
For those Windows users who wish to change IPMI server settings remotely, use SMCIPMITool for Windows.

Ensure your Command Prompt utility is pointing at the directory where the SMCIPMITool.exe file is located.

Here is a sample to set CPU fan @ 50%: SMCIPMITool.exe 192.168.0.100 myusername mypassword ipmi raw 30 70 66 01 00 32

Here is a sample to set Case/Chassis fans @ 50%: SMCIPMITool.exe 192.168.0.100 myusername mypassword ipmi raw 30 70 66 01 01 32

Green Value = CPU or Case/Chassis Fans (00 for CPU or 01 for Case/Chassis)

Blue Value = Fan Speed (Change value "32" to your taste. 00 = 0%, 16 = 25%, 32 = 50%, 64 = 100%. Nothing stop you from using other values inside the 00 - 64 scope. If you want it @ 75%, then use value 48.)

If you get "00" after issuing the command, your settings has been accepted by the server.

Hope this helps!
x9dri-ln4f returns:
Code:
CC
(Invalid data field in Request)
So yeah - looks like on this motherboard this function is really disabled :(
 

solon

Member
Apr 1, 2021
46
2
8
This is great stuff. I've been sporadically lurking on servethehome for quite a time, but just now registered. This seems to be one of the few places with decent info suitable for those of us who mess with server hardware as a sort of hobby project.

I recently acquired a Supermicro X10DRH-iT as a COVID keep-oneself-occupied project, on which this worked fine yesterday. I'm running ubuntu 20.10 (desktop). I'm using low power xeons on the board so the fans really hardly have to run at all. If I can't get the fans to slow down I guess I'll watercool it or something, so I was really happy with the information here about getting fan speed down.

I made a script that sets the profile to max with ipmi tool, sleeps 2s, then uses raw to set each fan zone to 25%. This was working fine yesterday.

This morning though, the fans are running at max, and running my script only sets the fans to 25% for a few seconds, then it goes back to max. I assume the BMC is overriding the 25% setting, but it was my understanding that the max profile before the raw commands should stop this from happening?

Has anyone by any chance seen similar behaviour on a X10 series supermicro board and maybe come up with a workaround?
 

weust

Active Member
Aug 15, 2014
345
34
28
42
Not having a X10 but a X11, I set the fan speed to max, then to n% per zone.
I do not use GNU/Linux, but on my FreeBSD server it looks like this and you should be able to use that too:
(below is part of my sh script I use at start up)

Code:
command="/usr/local/bin/ipmitool"

    echo -n "IPMI set fan mode to "full""
    $command raw 0x30 0x45 0x01 0x01
    sleep 1
    echo -n "IPMI set fans in "system" zone to 50% (front inlet fans)"
    $command raw 0x30 0x70 0x66 0x01 0x00 0x32
    echo -n "IPMI set fans in "peripheral" zone to 50% (CPU fan)"
    $command raw 0x30 0x70 0x66 0x01 0x01 0x32
The "sleep 1" is needed because it needs less then a second to set the zones to full speed.
Not using it will makes the fans go full right after the next two lines.
 

solon

Member
Apr 1, 2021
46
2
8
This looks almost the same as my script which I based off of something from this site somewhere. Yours is neater. Minor differences, I set everything to 25% and sleep for 2 seconds.

What mystifies me is that is was staying at 25% fine yesterday. Now, when I execute the script, after 2s it goes down to 25%, then after a few seconds ramps back up to 100%. If I run the script again almost right away, it goes to 25%, then almost instantly ramps back up to 100%.

The article on this site warns of this behaviour if the fan mode isn't set to max initially, but it seems that setting it to max doesn't help anymore, while it was behaving fine yesterday. I suppose I'll reboot later and see if it behaves differently, but it would be nice to understand why its happening.
 

weust

Active Member
Aug 15, 2014
345
34
28
42
I set mine so that the fans are quiet enough in the side room the server is in . 3U with 80mm Noctua fans.

Did you set max fan speed in the BMC as well? IIRC that is a requirement as well in this case.
My other server is a Dell R230 and different actions are needed there, so it can change per server/BMC.
 

solon

Member
Apr 1, 2021
46
2
8
I hadn't tried setting it to max in the BMC aswell. I just took a look, but it's now displaying as being set to max in the BMC too. I never set it to max there, but apparently it picks up the max setting issued from the ipmitool raw command and displays it in the web interface. Always nice when that works both ways. I've saved it as full in the BMC again for good measure and rerun the script. All I need to do now is go downstairs and see if it worked.

My X10 lives on a couple of bits of wood in a cabinet underneath the TV (until the EATX tray I ordered from our Chinese comrades comes in). The 2650LV3's only have a 65W TDP so 25% duty cycle is plenty. The cabinet itself has a 200mm fan to move warm air out of it. At 25% duty cycle it's quiet enough that you can't really hear it above the aquarium pump nearby. Still though... maybe I will watercool it anyway. Have had a half formed plan for a watercooling circuit throughout the house for some time now and since we're going in extra double plus lockdown again now anyway, perhaps it's a good time.
 

solon

Member
Apr 1, 2021
46
2
8
Well, I am officially stumped. Nothing makes any difference, the 'new' behaviour (that which wasn't happening yesterday) is consistent across reboots and changing the BMC to optimal, saving it, then changing it back to full and then saving it. All while the CPU packages are no hotter than 34C, which should exclude the possibility of any sort of override kicking in.

The way supermicro implemented this doesn't seem to be all that brilliant, even if you don't care about the noise, it's still potentially a waste of energy that should have been trivially easy to avoid. Oh well.

I may add a 200mm inlet fan to the cabinet and just remove the CPU fans. They don't seem to get any hotter than 45C that way either and at least it's quiet.
 

kodi

New Member
Aug 5, 2020
15
0
1
Well, I am officially stumped. Nothing makes any difference, the 'new' behaviour (that which wasn't happening yesterday) is consistent across reboots and changing the BMC to optimal, saving it, then changing it back to full and then saving it. All while the CPU packages are no hotter than 34C, which should exclude the possibility of any sort of override kicking in.

The way supermicro implemented this doesn't seem to be all that brilliant, even if you don't care about the noise, it's still potentially a waste of energy that should have been trivially easy to avoid. Oh well.

I may add a 200mm inlet fan to the cabinet and just remove the CPU fans. They don't seem to get any hotter than 45C that way either and at least it's quiet.
Just a hunch - are all fans running properly and are all in the defined zones? If just one fan is not running properly (disconnected, out of the defined RPM zone) all fans will switch to full speed.
 

solon

Member
Apr 1, 2021
46
2
8
I eventually found the the behaviour cleared up by resetting the BMC. Not sure how it could get into such a strange loop. I've since added another 200mm fan that pulls air into the cabinet and I'm attempting to 3d print a duct that will fit over both coolers (using this style: supermicro SNK-P0050AP4), and mount a single 200mm fan to draw air through both heatsinks, assuming the print ever finishes without coming unstuck halfway.

fanhouders_3.png

The noctua fans I'm using can be limited to 550rpm, which really should make the whole setup almost completely silent and stop strange BMC behaviour from being able to influence that. I can't really imagine anyone has the same set of problems that the design shown could be of use to anyone other than me, but in the unlikely event that someone else wants this solution I cam make openscad or stl files available.
 

solon

Member
Apr 1, 2021
46
2
8
So my BCM really doesn't behave in a reliable way. It will behave normally for a while after reboot, then when there is a load it will ramp up fan speeds and never ramp them back down again. Resetting the BCM sometimes works to get back to normal behaviour, but even that isn't consistent.

I've given up on using the BCM for anything and have just used 2x 200mm noctua PWM fans with the supplied resistor to limit to 550rpm max. I got a singlt 120mm 17db noctua van for the icydock 5x3.5 in 3x5.25 cage with another 3d printed adapter and everything is now sufficiently silent, though since the BCM eventually sets everything as high as it will go and keeps it there, it usually still isn't completely silent.

I may try the using the secondary fan bank trick too, that could result in lower rpms more regularly or if it continues to annoy me I may simply watercool the CPU's instead in the end, but I think I'll check around for a standalone seperate fan controller first.
 

David Reitz

New Member
Jul 23, 2019
2
1
3
Working on an X10SRH board with the latest IPMI. Just a note, if you're using slow fans (like the fans on the Noctua NH-U9DX i4 I'm using) and set the duty cycle to something low, they'll drop below the lower critical threshold and the IPMI will spin them right back up again. Just lower the threshold to below whatever your fans are at to prevent that. (e.g. ipmitool sensor thresh FAN1 lower 100 150 200)
Thank you! This worked perfectly with the fans connected to my Supermicro X10SLM-F motherboard.
 

testcb00

New Member
Feb 26, 2021
4
1
3
These two work on my x9srw that has IPMI fw 3.39.
ipmitool raw 0x30 0x91 0x5A 0x3 0x10 0x7f
ipmitool raw 0x30 0x91 0x5A 0x3 0x11 0x7f

0xff is 100% duty cycle. 0x7f is 50%.

I also have a x9dri-ln4f+ and a x9drw-f. Neither the above two commands nor
ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x32
work. I would be eternally grateful if anybody could point me to the right IPMI fw version and right raw commands for these boards.
My X9SRG-F can use
Code:
ipmitool raw 0x30 0x91 0x5A 0x3 0x10 0x7f
ipmitool raw 0x30 0x91 0x5A 0x3 0x11 0x7f
ipmitool raw 0x30 0x91 0x5A 0x3 0x12 0x7f
To control the FAN
0x10 for FAN 1-4
0x11 for FANA FANB
0x12 for FANC FAND
0x7f can be smaller number (0-255) 255 = 0xff
 
  • Like
Reactions: dawsonkm

mk01

New Member
Oct 3, 2021
1
1
3
@kodi, @Sonner

ipmitool raw 0x30 0x91 0x5A 0x3 0x10 0x7f

works on x9dri-ln4f+ with IPMI 3.61 (dated 2020-01-07).

also, some guys were asking about x9drd-???. according the Supermicro FAQ answers, this functionality of direct control was removed from IPMI firmware. I tried firmwares down to 3.15 and none worked, then I gave up on trying to set this via raw ipmi commands.

BUT: I had success with controlling them via linux hwmon driver for the chip on this MB models which is nct7904. after loading it creates /sys/class/hwmon/hwmonX/ folder with pwm1 and pwm1_enable files (it is still needed to set fan mode to full before as with the other described solutions).
Although this way works, it has quite irritating issue - the chip is being accessed via i2c and communication via it requires locking control. and because that lock is not being handled at hw level, so driver and ipmi basically happen to clash while reading/writing which kicks the driver out of control reporting ~"communication errors" by i2c-i801 in kernel log. then removal of nct7904, i2c-i801, i2c-smbus and loading them back was needed.
I managed to minimise this hassle with various settings, like for instance disabling readouts (temps, in, ...) - all but the fan_inputs by lm_sensors and not trying to set the pwm more often than once / 30s.
All this being said, for x9drd I ended this story by buying Corsair Commander PRO controller (there is a driver in linux for it, it presents 6pwm controlled outputs/inputs and 4temp inputs). I burned two until I realised that the controller does not meet the paper specs (1A max per pwm / 4A in total) ((the SM boards are on the other hand excelent with this, in general the fan headers are rated for 3A each)). So finally I had to attach my fans to it with cable providing external power (molex/sata).

Anyhow a bit of info to question whether those boards provide controll per fan header - the answer is no. The limitation is set by the nct7904 which can monitor up to 12 fans,. but it has only 4pwm controlled outputs, so SM is wiring them into the usual zones (system only - 1 zone, or system + cpu - 2 zones). Also what I realised is, that not even the IPMI fw is controlling the fans directly, it uses the auto controll provided by the nct7904 which allows to link those 4pwm to various inputs (temp/rpm/peci) and various alarms (high temp, low rpm etc). the profile tables are fed on power-on of the MB by nct7904 itself from some storage (perhaps BIOS?????). they can be reprogrammed on the fly and I initially planned to do that, but all this takes so much time, ... then I bought the CCPro and never looked back.

Last note to the issues in respect to "ipmi reverting back the controls" - the low RPM of a fan is far not the only factor here. as said in the previous paragraph, the nct auto control is linked to various temps too - for sure CPU/PCH and maybe even the DIMMs (as peci can read those and all this data are then pushed to nct). One way is the already provided solution with changing the assertion levels, but perhaps easier is to simply turn the assertions off.

export the sensor setup with:
ipmi-sensors-config --checkout --filename=ipmi-sensors-config

edit the file by changing "Yes" on the required assertions to "No" and load the file back:
ipmi-sensors-config --commit --filename=ipmi-sensors-config

One of my systems is x9dri board inside SC847 case with all 36 drive bays full, another 8x 2.5" SSDs inside (the case is 4U, 24bays front, 12bays back. due to the 12bays in back/bottom, the MB department is only low profile so basically the back is split into 2x2U and there was lot of space for the ssd drives between the middle FANs and back Backplane) + 6xNVME drives in 3x aoc-slg3-2m2 addon cards. The board is fully loaded with 24dimms.
To cool that beast I had to swap the factory passive cpu coolers for active ones and two more screamers (FAN-0127L4) were needed, standing behind each other to double the air pressure (in the lower department right in front of the back backplane). SC847 fully loaded like this is making any Noctuas useless unfortunatelly (far too low pressure), honestly it renders any consumer grade FAN useless. So if anyone complains about the noise, imagine to have instead of 7 provided middle fans in the SC847, 11 of those.
So after many weeks struggling with the noise and my marriage going towards divorce I decided for a dramatic rebuild.
Spoiler alert - I ended up with custom water loop consisting of two radiators (one 2x12cm wide, one 3x12cm wide), two cpu water blocks and two Peltiers. this made it possible to run only with 3 middle fans at lowest rpm (1400).
The result is: "wife approved"

@solon
I wish I had 3D printer. the rebuild would be a lot easier. Kudos anyhow for that idea, I think 3D printer should be required tool for any "do it yourself" guy.
 
Last edited:
  • Like
Reactions: dawsonkm

solon

Member
Apr 1, 2021
46
2
8
In the end I did figure out that the setting the fans to max thing had to do with the rpm range the ipmi expected to see. Obviously, the 200mm fan has rather low rpm's and the reaction of the ipmi was to set everything to max... sort of at some point when the rpm's were outside the expected range. It's been working very well for a while now. Great thread.
 

peter_s

New Member
Oct 15, 2021
2
0
1
Based on several references and projects (on this site, on TrueNAS forums, and on different blogs) I just created a systemd service for Linux to control my fans through IPMI on my Super Micro X11SCH-F motherboard. This service switches to FULL fan mode and can control the fans based on temperature of CPU and HD zones, separately. You can take a look at here:

GitHub - petersulyok/smfc: Super Micro Fan Control

Any feedback is appreciated and big thanks for community for collecting and sharing information on IPMI and Super Micro motherboards, without this I wouldn't have been able to make this.
 

fl0w

Member
Oct 28, 2015
65
22
8
35
Paris
Based on several references and projects (on this site, on TrueNAS forums, and on different blogs) I just created a systemd service for Linux to control my fans through IPMI on my Super Micro X11SCH-F motherboard. This service switches to FULL fan mode and can control the fans based on temperature of CPU and HD zones, separately. You can take a look at here:

GitHub - petersulyok/smfc: Super Micro Fan Control

Any feedback is appreciated and big thanks for community for collecting and sharing information on IPMI and Super Micro motherboards, without this I wouldn't have been able to make this.
Hi, is there a way to define for which area each fan header is running?