P

Supermicro X9/X10/X11 Fan Speed Control

weust

Active Member
Aug 15, 2014
323
31
28
40
A while back I made a very small script for my FreeBSD ZFS server.
BMC is set to full speed, then the script sets the two fan zones on my motherboard to a certain percentage.
It needs 1 second between lines otherwise it will mess things up.

The script is placed in /usr/local/etc/rc.d/ iirc, so I can enable it in /etc/rc.conf.

What I would like to figure out next, with my limited shell scripting knowledge+the internet, is to create a cron job that will check the temperature of the disks, CPU, etc and adjust the fanspeed accordingly.
Increase or decrease by 5% speed based on 5C temperature increase/decrease. Something like that.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,218
412
83
In the linux universe at least, this was achievable with fancontrol/pwmconfig (although I don't think to the extent of 5% increments*), I'm not sure if FreeBSD has anything to match (wikipedia only mentions OpenBSD or Dragonfly in the context of userspace fan control).

* The settings I used way back when was basically set a score based on a combination of CPU and HDD temps, and then fiddle the PWM values directly. By the time I found hddfancontrol did the same thing but better, I'd moved to IPMI motherboards where the BIOS made a good enough job of it.
 

weust

Active Member
Aug 15, 2014
323
31
28
40
The 5% was more an idea. And if I can find some software that does it all for me, even better.
Saves me time :)
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,218
412
83
I assume you're using a Supermicro motherboard and using the raws to control the duty cycle?

What you'll want to do is cycle through your hard drives (I assume you'll have to use smartctl as I don't think you'll have hddtemp available) and calculate either the peak or average temperature and run your duty cycle parameters accordingly. Here's a very quick'n'dirty way of doing this in bash on linux (don't think BSD uses /sys/block so adapt as necessary) with a dependency on bc to do some of the arithmetic:
Code:
#!/bin/bash
discs=($(find /sys/block -maxdepth 1 -type l -iname 'sd*' -printf '%P '))

all_disc_temps=0
max_disc_temp=0

for disc in "${discs[@]}";
do
    disc_temp=$(smartctl -a /dev/"${disc##*/}"|awk '/Temperature_Celsius/{print $10}')
    all_disc_temps=$(echo "$disc_temps + $disc_temp"|bc)
    if [[ $disc_temp -gt $max_disc_temp ]]; then
        max_disc_temp=$disc_temp
    fi
done

avg_disc_temp="$(echo "$all_disc_temps / ${#discs[@]}"|bc)"

echo "Maximum disc temperature over ${#discs[@]} discs is $max_disc_temp"
echo "Average disc temperature over ${#discs[@]} discs is $avg_disc_temp"
(Seriously, the above code won't work directly on BSD and has no error checking so don't be surprised if it doesn't work and makes your server explode. It makes a lot of assumptions and heavily abuses bash's lack of typing, very hacky, YMMV, BOGOF, £0.02, etc)

The above produces something like this on my current machine:
Code:
root@wug:~# /scratch/smart_temps
Maximum disc temperature over 9 discs is 43
Average disc temperature over 9 discs is 37
I don't think there's much benefit in running it every minute (and indeed if you follow my example it's not a fast script to run as it queries each drive sequentially - s'one of the reasons I usually use hddtemp to do it since you can grab all temperatures over a local socket simultaneously and don't need to be root to do it; on a system with a lot of drives the above could conceivably take more than a minute to run anyway) but you could easily add some if blocks for various levels of fan duty cycle at whatever thresholds you want to set e.g. go to maximum duty cycle if the drive temperature exceeds 50. Then, every 5mins, the script will run and adjust the duty cycle accordingly.
 
Last edited:

weust

Active Member
Aug 15, 2014
323
31
28
40
I'll have a look someday when I have time and when it rains.
Also I like to avoid bash and prefer plain sh.

FreeBSD has smartctl, so I can grep the temps from there.

Doing it on a disk temp level is a good idea.
Hadn't thought that far ahead yet.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,218
412
83
I'd say that if you've got bash available, and especially if you're not au fair with scripting, then use it over sh. Plain sh is generally missing a huge amount of terribly useful functionality (depending on which exact sh you're actually using) that makes life much easier for the casual scripter. Much of the script above would be doable without casting into arrays I think but I find arrays, parameter expansion and scoping far too useful to give up.

As long as you can get the various temperatures you care about into variables that can be treated as numbers, it's pretty trivial to just run them through a set of if statements or a case block to set whatever duty cycle you deem appropriate for that temperature. That'll depend on how your systems are set up, what fans are plugged in to where, what zones you have available in your IPMI, etc - you can start getting into relatively complicated logic quite quickly, so I'd just advise trying a one-size-fits-all approach first.
 

weust

Active Member
Aug 15, 2014
323
31
28
40
bash is available as either a pkg or in ports, I just don't like to install it.
I like to keep things as GNU-free as possible.

I use tcsh as my user shell. The sh is just the one that comes with FreeBSD.

I am not very experienced in shell scripting (PowerShell is more my thing) but I will give it a go.
Will see how far I can get.
 

Falloutboy

Member
Oct 23, 2011
117
13
18
PigLover submitted a new resource:

Supermicro X9/X10/X11 Fan Speed Control - Supermicro fan speed control

Read more about this resource...
Something is a bit off with regard to this and my X10DRX, looking at the motherboard manual on page 1-3 it shows Fan6 as being CPU1 and Fan7 as CPU2.
At the moment just to get everything up and running this is under Windows 10 Pro with the Superdoctor software, after I have fully validated all the hardware it will be switched to Linux but this is a good time to be learning about such things.. Is the information you mentioned only relevant when read via the BMC?
 

anomaly

Active Member
Jan 8, 2018
189
38
28
Bummer, looks like I have to set them at full blast, or control all of them per zone. Better than nothing, but still.
 

AndrewX192

Member
Apr 10, 2016
57
23
8
28
I stumbled across this post while trying to get the fans under control in my SuperMicro SYS-1019S-WR with X11SSW-F. Unfortunately, when I set the fan mode to "full" and set the system fan targets to 37.5% (e.g. via ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x24), the fan speed is only controlled for a second (or less), and the system immediately takes over and ramps the fans back up

I don't see why this would be necessary, as my CPU temps are around 60c under full load and my fan thresholds are well above what they need:

CPU Temp | 59.000 | degrees C | ok | 0.000 | 0.000 | 5.000 | 95.000 | 100.000 | 100.000
PCH Temp | 32.000 | degrees C | ok | 0.000 | 5.000 | 10.000 | 90.000 | 95.000 | 100.000
System Temp | 21.000 | degrees C | ok | -10.000 | -5.000 | 0.000 | 80.000 | 85.000 | 90.000
Peripheral Temp | 28.000 | degrees C | ok | -10.000 | -5.000 | 0.000 | 80.000 | 85.000 | 90.000
VcpuVRM Temp | 33.000 | degrees C | ok | -5.000 | 0.000 | 5.000 | 95.000 | 100.000 | 105.000
DIMMA1 Temp | na | | na | na | na | na | na | na | na
DIMMA2 Temp | na | | na | na | na | na | na | na | na
DIMMB1 Temp | na | | na | na | na | na | na | na | na
DIMMB2 Temp | na | | na | na | na | na | na | na | na
FAN1 | 4000.000 | RPM | ok | 300.000 | 500.000 | 700.000 | 25300.000 | 25400.000 | 25500.000
FAN2 | 4400.000 | RPM | ok | 300.000 | 500.000 | 700.000 | 25300.000 | 25400.000 | 25500.000
FAN3 | 4500.000 | RPM | ok | 300.000 | 500.000 | 700.000 | 25300.000 | 25400.000 | 25500.000
FAN4 | na | | na | na | na | na | na | na | na
FAN5 | 4200.000 | RPM | ok | 300.000 | 500.000 | 700.000 | 25300.000 | 25400.000 | 25500.000
FAN6 | na | | na | na | na | na | na | na | na
12V | 12.126 | Volts | ok | 10.173 | 10.299 | 10.740 | 12.945 | 13.260 | 13.386
5VCC | 5.000 | Volts | ok | 4.246 | 4.298 | 4.480 | 5.390 | 5.546 | 5.598
3.3VCC | 3.350 | Volts | ok | 2.789 | 2.823 | 2.959 | 3.554 | 3.656 | 3.690
VBAT | 3.074 | Volts | ok | 2.407 | 2.494 | 2.610 | 3.509 | 3.596 | 3.712
Vcpu | 1.083 | Volts | ok | 0.111 | 0.111 | 0.111 | 1.596 | 1.758 | 1.776
VDIMMAB | 1.191 | Volts | ok | 0.948 | 0.975 | 1.047 | 1.344 | 1.425 | 1.443
0.95V VCCIO | 0.978 | Volts | ok | 0.870 | 0.897 | 0.942 | 1.194 | 1.221 | 1.248
1.5VSB | 1.536 | Volts | ok | 1.320 | 1.347 | 1.401 | 1.644 | 1.671 | 1.698
5VSB | 4.922 | Volts | ok | 4.246 | 4.298 | 4.480 | 5.390 | 5.546 | 5.598
3.3VSB | 3.248 | Volts | ok | 2.789 | 2.823 | 2.959 | 3.554 | 3.656 | 3.690
1.05V VCCSA | 1.059 | Volts | ok | 0.861 | 0.888 | 0.960 | 1.086 | 1.149 | 1.176
1.2V BMC | 1.218 | Volts | ok | 1.020 | 1.047 | 1.092 | 1.344 | 1.371 | 1.398
1.0V PCH | 1.005 | Volts | ok | 0.870 | 0.897 | 0.942 | 1.194 | 1.221 | 1.248
Chassis Intru | 0x0 | discrete | 0x0000| na | na | na | na | na | na
PW Consumption | 75.000 | Watts | ok | na | na | na | na | na | na
PS1 Status | 0x9 | discrete | 0x0900| na | na | na | na | na | na
PS2 Status | 0x1 | discrete | 0x0100| na | na | na | na | na | na
 

StevenDTX

Active Member
Aug 17, 2016
407
137
43
I stumbled across this post while trying to get the fans under control in my SuperMicro SYS-1019S-WR with X11SSW-F. Unfortunately, when I set the fan mode to "full" and set the system fan targets to 37.5% (e.g. via ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x24), the fan speed is only controlled for a second (or less), and the system immediately takes over and ramps the fans back up

I don't see why this would be necessary, as my CPU temps are around 60c under full load and my fan thresholds are well above what they need:

CPU Temp | 59.000 | degrees C | ok | 0.000 | 0.000 | 5.000 | 95.000 | 100.000 | 100.000
PCH Temp | 32.000 | degrees C | ok | 0.000 | 5.000 | 10.000 | 90.000 | 95.000 | 100.000
System Temp | 21.000 | degrees C | ok | -10.000 | -5.000 | 0.000 | 80.000 | 85.000 | 90.000
Peripheral Temp | 28.000 | degrees C | ok | -10.000 | -5.000 | 0.000 | 80.000 | 85.000 | 90.000
VcpuVRM Temp | 33.000 | degrees C | ok | -5.000 | 0.000 | 5.000 | 95.000 | 100.000 | 105.000
DIMMA1 Temp | na | | na | na | na | na | na | na | na
DIMMA2 Temp | na | | na | na | na | na | na | na | na
DIMMB1 Temp | na | | na | na | na | na | na | na | na
DIMMB2 Temp | na | | na | na | na | na | na | na | na
FAN1 | 4000.000 | RPM | ok | 300.000 | 500.000 | 700.000 | 25300.000 | 25400.000 | 25500.000
FAN2 | 4400.000 | RPM | ok | 300.000 | 500.000 | 700.000 | 25300.000 | 25400.000 | 25500.000
FAN3 | 4500.000 | RPM | ok | 300.000 | 500.000 | 700.000 | 25300.000 | 25400.000 | 25500.000
FAN4 | na | | na | na | na | na | na | na | na
FAN5 | 4200.000 | RPM | ok | 300.000 | 500.000 | 700.000 | 25300.000 | 25400.000 | 25500.000
FAN6 | na | | na | na | na | na | na | na | na
12V | 12.126 | Volts | ok | 10.173 | 10.299 | 10.740 | 12.945 | 13.260 | 13.386
5VCC | 5.000 | Volts | ok | 4.246 | 4.298 | 4.480 | 5.390 | 5.546 | 5.598
3.3VCC | 3.350 | Volts | ok | 2.789 | 2.823 | 2.959 | 3.554 | 3.656 | 3.690
VBAT | 3.074 | Volts | ok | 2.407 | 2.494 | 2.610 | 3.509 | 3.596 | 3.712
Vcpu | 1.083 | Volts | ok | 0.111 | 0.111 | 0.111 | 1.596 | 1.758 | 1.776
VDIMMAB | 1.191 | Volts | ok | 0.948 | 0.975 | 1.047 | 1.344 | 1.425 | 1.443
0.95V VCCIO | 0.978 | Volts | ok | 0.870 | 0.897 | 0.942 | 1.194 | 1.221 | 1.248
1.5VSB | 1.536 | Volts | ok | 1.320 | 1.347 | 1.401 | 1.644 | 1.671 | 1.698
5VSB | 4.922 | Volts | ok | 4.246 | 4.298 | 4.480 | 5.390 | 5.546 | 5.598
3.3VSB | 3.248 | Volts | ok | 2.789 | 2.823 | 2.959 | 3.554 | 3.656 | 3.690
1.05V VCCSA | 1.059 | Volts | ok | 0.861 | 0.888 | 0.960 | 1.086 | 1.149 | 1.176
1.2V BMC | 1.218 | Volts | ok | 1.020 | 1.047 | 1.092 | 1.344 | 1.371 | 1.398
1.0V PCH | 1.005 | Volts | ok | 0.870 | 0.897 | 0.942 | 1.194 | 1.221 | 1.248
Chassis Intru | 0x0 | discrete | 0x0000| na | na | na | na | na | na
PW Consumption | 75.000 | Watts | ok | na | na | na | na | na | na
PS1 Status | 0x9 | discrete | 0x0900| na | na | na | na | na | na
PS2 Status | 0x1 | discrete | 0x0100| na | na | na | na | na | na


did you reboot the IPMI after you made the change? That’s required.
 

AndrewX192

Member
Apr 10, 2016
57
23
8
28
did you reboot the IPMI after you made the change? That’s required.
For my most recent test, I set the fan speeds to "Full" in the IPMI web interface, and then did an ipmitool mc reset cold. Once the BMC came back up (e.g. I could get a list of sensors and their values), I set the system fan speed targets to 37.5%, the change only applied for a few seconds before the system took over.
 

weust

Active Member
Aug 15, 2014
323
31
28
40
This is odd. I have a X11SSi-LN4F-O motherboard, and set the fans to full (iirc) then use ipmitool to adjust it.
Works fine. I will check tonight to be sure on the BMC setting for the fan speed setting.
 

weust

Active Member
Aug 15, 2014
323
31
28
40
Looked it up, and the way I did it was no doubt using the script from this topic, I think?
It first sets the IPMI fan speed mode to "full" and then sets the zones to 50%.
The full script is as follows:

Code:
#!/bin/sh

# LOGIN

. /etc/rc.subr

name=setfanspeed
rcvar=setfanspeed_enable

start_cmd="${name}_start"
stop_cmd=":"

: ${setfanspeed_enable:="No"}

command="/usr/local/bin/ipmitool"

setfanspeed_start()
{
    echo -n "IPMI set fan mode to "full""
    $command raw 0x30 0x45 0x01 0x01
    sleep 1
    echo -n "IPMI set fans in "system" zone to 50% (front inlet fans)"
    $command raw 0x30 0x70 0x66 0x01 0x00 0x32
    echo -n "IPMI set fans in "peripheral" zone to 50% (CPU fan)"
    $command raw 0x30 0x70 0x66 0x01 0x01 0x32
    }

load_rc_config $name
run_rc_command "$1"
Save this to a file in /usr/local/etc/rc.d/ chmod to have it executable and add it as a service.

Hope this helps you.
 

AndrewX192

Member
Apr 10, 2016
57
23
8
28
Looked it up, and the way I did it was no doubt using the script from this topic, I think?
It first sets the IPMI fan speed mode to "full" and then sets the zones to 50%.
The full script is as follows:

Code:
#!/bin/sh

# LOGIN

. /etc/rc.subr

name=setfanspeed
rcvar=setfanspeed_enable

start_cmd="${name}_start"
stop_cmd=":"

: ${setfanspeed_enable:="No"}

command="/usr/local/bin/ipmitool"

setfanspeed_start()
{
    echo -n "IPMI set fan mode to "full""
    $command raw 0x30 0x45 0x01 0x01
    sleep 1
    echo -n "IPMI set fans in "system" zone to 50% (front inlet fans)"
    $command raw 0x30 0x70 0x66 0x01 0x00 0x32
    echo -n "IPMI set fans in "peripheral" zone to 50% (CPU fan)"
    $command raw 0x30 0x70 0x66 0x01 0x01 0x32
    }

load_rc_config $name
run_rc_command "$1"
Save this to a file in /usr/local/etc/rc.d/ chmod to have it executable and add it as a service.

Hope this helps you.
I tried running those three commands with a second of time between each, but the result is the same - the system returns to full fan speeds after a second or two, even when the system is idle. My BMC firmware is latest, but my BIOS is a revision behind.

Do you have all the fans in your system populated? I have 6 fan headers (FAN1-FAN6), of which only FAN1-FAN4 are actually connected. I'm going to look at adding two more Noctua 40x20 fans, and maybe replacing the 4x counter rotating fans, at least for testing purposes.
 

weust

Active Member
Aug 15, 2014
323
31
28
40
One header isn't used. Three are intake and one is for the CPU fan.
Nothing special really.
 

AndrewX192

Member
Apr 10, 2016
57
23
8
28
I tried installing a second power supply (to meet the redundancy it was expecting), doing a full factory reset / defaults on IPMI, and trying to control the fan speeds again. When the system is in "Full" mode, it will still override any fan options within 1-2 seconds of the setting being set.
 

weust

Active Member
Aug 15, 2014
323
31
28
40
Mine came with two PSU's but I replaced them with a single one that makes less noise, so I know it should work.

To give you an idea of my system, this is an overview of ipmitool sensor:
FANA is the CPU.

Code:
% doas ipmitool sensor
CPU Temp         | 32.000     | degrees C  | ok    | 0.000     | 0.000     | 0.000     | 95.000    | 100.000   | 100.000
PCH Temp         | 37.000     | degrees C  | ok    | 0.000     | 5.000     | 16.000    | 90.000    | 95.000    | 100.000
System Temp      | 30.000     | degrees C  | ok    | -10.000   | -5.000    | 0.000     | 80.000    | 85.000    | 90.000
Peripheral Temp  | 47.000     | degrees C  | ok    | -10.000   | -5.000    | 0.000     | 80.000    | 85.000    | 90.000
VcpuVRM Temp     | 37.000     | degrees C  | ok    | -5.000    | 0.000     | 5.000     | 95.000    | 100.000   | 105.000
DIMMA1 Temp      | 31.000     | degrees C  | ok    | -5.000    | 0.000     | 5.000     | 80.000    | 85.000    | 90.000
DIMMA2 Temp      | 33.000     | degrees C  | ok    | -5.000    | 0.000     | 5.000     | 80.000    | 85.000    | 90.000
DIMMB1 Temp      | 32.000     | degrees C  | ok    | -5.000    | 0.000     | 5.000     | 80.000    | 85.000    | 90.000
DIMMB2 Temp      | 32.000     | degrees C  | ok    | -5.000    | 0.000     | 5.000     | 80.000    | 85.000    | 90.000
FAN1             | 900.000    | RPM        | ok    | 300.000   | 500.000   | 700.000   | 25300.000 | 25400.000 | 25500.000
FAN2             | 1000.000   | RPM        | ok    | 300.000   | 500.000   | 700.000   | 25300.000 | 25400.000 | 25500.000
FAN3             | 900.000    | RPM        | ok    | 300.000   | 500.000   | 700.000   | 25300.000 | 25400.000 | 25500.000
FAN4             | na         |            | na    | na        | na        | na        | na        | na        | na
FANA             | 1500.000   | RPM        | ok    | 300.000   | 500.000   | 700.000   | 25300.000 | 25400.000 | 25500.000
12V              | 12.448     | Volts      | ok    | 10.144    | 10.272    | 10.784    | 12.960    | 13.280    | 13.408
5VCC             | 5.078      | Volts      | ok    | 4.246     | 4.298     | 4.480     | 5.390     | 5.546     | 5.598
3.3VCC           | 3.401      | Volts      | ok    | 2.789     | 2.823     | 2.959     | 3.554     | 3.656     | 3.690
VBAT             | 2.823      | Volts      | nc    | 2.636     | 2.738     | 2.857     | 3.843     | 3.945     | 4.064
VCPU             | 0.543      | Volts      | ok    | na        | na        | na        | 1.585     | 1.752     | 1.775
VCCSA            | 1.330      | Volts      | ok    | 1.054     | 1.102     | 1.150     | 1.546     | 1.594     | 1.630
VDIMMABCD        | 1.191      | Volts      | ok    | 0.948     | 0.975     | 1.047     | 1.344     | 1.425     | 1.443
5VSB             | 5.081      | Volts      | ok    | 4.244     | 4.298     | 4.487     | 5.378     | 5.540     | 5.594
3.3VSB           | 3.356      | Volts      | ok    | 2.634     | 2.748     | 2.843     | 3.850     | 3.945     | 4.059
Chassis Intru    | 0x1        | discrete   | 0x0100| na        | na        | na        | na        | na        | na
PS1 Status       | 0x1        | discrete   | 0x0100| na        | na        | na        | na        | na        | na