Reference Material Supermicro X9/X10/X11 Fan Speed Control

Reference material regarding fan speed control on SuperMicro X9/X10/X11 motherboards. This has been pieced together over some time and I finally got that "aha!" moment yesterday where it all came together. A bit of testing confirmed that this is effectively correct (even if SM might disagree on some fine details).

Fans on these boards are controlled in two zones:

• CPU or System fans, labelled with a number (e.g., FAN1, FAN2, etc.) - zone 0
• Peripheral zone fans, labelled with a letter (e.g., FANA, FANB, etc.) - zone 1

On these boards there are 4 levels of speed control:

• Standard: BMC control of both fan zones, with CPU zone based on CPU temp (target speed 50%) and Peripheral zone based on PCH temp (target speed 50%)
• Optimal: BMC control of the CPU zone (target speed 30%), with Peripheral zone fixed at low speed (fixed ~30%)
• Full: all fans running at 100%
• Heavy IO: BMC control of CPU zone (target speed 50%), Peripheral zone fixed at 75%

You can set these speeds using IPMI.

Raw speed controls:

Here's the more interesting part - there are IPMI "raw" controls that you can use too! This section assumes you have loaded and use "ipmitool", but you can also use the GNU ipmitools or SM's ipmicfg for the same result. The incantation to use raw mode might be different - but the magic numbers are the same.

You can read the fan speed mode using the command:

The values are:

• Standard: 0
• Full: 1
• Optimal: 2
• Heavy IO: 4

I have no idea why they skipped "3"...

Here's where it gets more interesting:

This part is what I discovered from a post in the FreeNAS forum (reference below).

You can also set the PWM duty cycle for each of the fan zones. PWM values are set 64 steps using hex values from 00-FF (ox00 to 0x64). 0x00 is minimum speed, 0x64 is full speed, and 0x32 is 50%, etc. Each "zone" is set seperately using the following command:

ipmitool raw 0x30 0x70 0x66 0x01 0x<z> 0x<n>
- where "z" is the zone (0 0r 1)
- where "n" is the duty cycle (0x00..0x64)

Example: Set CPU zone (0) to 50%:
Example: Set Perpipheral zone (1) to 25%:
Really important note:

If you have the fan mode set to "Full" before setting the duty cycle them the BMC doesn't seem to ever change the fans. I suppose its possible that if you go above the high threshold values then it might alarm and reset them - but in normal operation they seem to be stable.

If you have the fan mode set to "Standard", "Optimal" or "Heavy IO" then the BMC will continue to try and manage the fans and will override your settings within a couple of minutes.

Final example:

For now, until I do something more sophisticated, I've added this to the script I run at boot on my 5-node Proxmox cluster. Its holding fan speeds much quieter and temps seem to be OK so far:

Code:

#set fan mode to "full"
ipmitool raw 0x30 0x45 0x01 0x01
#set fans in "system" zone to 37.5%
ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x24
#set fans in "peripheral" zone to 25%
ipmitool raw 0x30 0x70 0x66 0x01 0x01 0x16

References:

I first found the duty cycle info here. This post includes a pretty complete PIM-based script on the 4th page.
Script to control fan speed in response to hard drive temperatures | FreeNAS Community

Supermicro FAQ reference on the fan mode settings (this one leaves out "Heavy IO").
Super Micro Computer, Inc.. - FAQ Entry