SuperMicro X9SRL-F IPMI fan speed assistance

Hi All,

I've been doing a bit of reading on how to use impitool to adjust the fan thresholds on my motherboard, to prevent the spin up/down issue that some people have previously reported.

The basic steps I took were as per Decrease Supermicro IPMI Fan Threshold

This MB has 5 headers, FAN 1, 2, 3, 4 and FAN A. In my chassis, I have my CPU cooler fan on FAN 1, 2x 120mm chassis fans on FAN 2 and 3, a 90mm chassis fan on FAN 4 and another 120mm fan on FAN A.

What I am finding is, with my thresholds all set the same as per the above mentioned link, if I have the 90mm fan connected at all, it reintroduces the cycle up/down issue I was seeing before.
I suspect this is because this fan has different RPM limits compared to the 120mm fans, but I can't wrap my head around how to adjust the thresholds accordingly so that I can have this connected with lower than default thresholds and prevent the spinup/down issue.

Can anyone assist me in tweaking the IPMI thresholds to accomplish this?

Thanks
Eds

 

Specs of the Fans used?

 

All of these fans are from the Noctua Redux line here: Fans - Products
120mm ones are these: NF-S12B redux-1200 PWM
I think my exhausts are actually 80mm and are NF-R8 redux-1800 PWM

 

Is IPMI periodically reading the fan speed as 0rpm like wot mine did here?

 

They don't ever report 0rpm.
They show as the lower bound for these fans, which is about 375 rpm, then after couple of seconds they ramp up to max speed which is about 1200rpm and 1950rpm, then a couple of seconds they lower back to 3675 ish.

Running ipmitool sensor output confirms this every time I run it.
Fan mode is set to optimal, and this only occurs when the higher rpm 80mm fan is connected. As soon as that is disconnected, it stops :S

 

Is the 92mm fan a 3-pin rather than a 4-pin PWM model? I recall someone else on the forums experiencing the min/max cycling issue when they had a 3-pin fan attached to the fan headers.

 

No all the fans are 4 pin PWM fans.

I did have 2 of the 80mm fans on a 4 pin to 4/3 pin splitter which I thought was the problem, but that wasn't the case as I connected just one of them directly to the header and it still does it.

Definitely feels like a threshold issue, but none of the lower thresholds are ever breached.

 

Hmm well the only behaviour I observed was the occasional dropping to 0rpm right before an assert/fan "surge". My problem was that, when they went below their 300rpm threshold the PWM sensor reported at 0rpm; it's possible your fans behave slightly differently but still suffer from the same basic problem of not reporting properly once they fall below their "stall" speed. Does the IPMI log say there's been a fan assert? IIRC it's visible in the web GUI someplace.

Have you tried setting the fan mode to "full" rather than optimal and tweaking the duty cycle via the raws?

 

IIRC, setting to full speed runs them at full all the time without issue, but I don't really understand the concept of the duty cycle enough to tinker with this, so not really sure what I'm doing there.

I can't be certain, but I think if I set it to standard, they all run at lowest RPM without issue. I will have to confirm this when I get home.

On a side note, do any of these settings changes get reset to default when the board is physically unplugged from power?
I just noticed my CPU temp upper thresholds are back to the defaults of 95 98 100, even though I lowered them previously to 60 70 80.

 

Not entirely surprised at your trepidation since it's rather cryptic at best and not terribly well documented (and the raw values might differ slightly between X8/X9 boards vs. the X10/X11 boards I think), but if you've not seen it already Pig Lover wrote a very readable intro to it here:
https://forums.servethehome.com/index.php?resources/supermicro-x9-x10-x11-fan-speed-control.20/

Otherwise, searching for supermicro+ipmi+raw will reveal a bevvy of people scratching their heads similarly. There's a scattering of not-easy-to-stumble-across-unless-you-know-they're-there FAQs on the supemicro site with similar details in them as well, e.g.
FAQ Entry | Online Support | Support - Super Micro Computer, Inc.
FAQ Entry | Online Support | Support - Super Micro Computer, Inc.
FAQ Entry | Online Support | Support - Super Micro Computer, Inc.

Basically the last hex digit in this command:

Code:

ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x64

...governs the duty cycle. 64 is the maximum/100%, so you can force e.g. 50% by halving the number to 32;

Code:

ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x32

IIRC that only has any effect when your fan profile it set to "full" but I could be mis-remembering.

I think if you set custom duty cycles and some of the thresholds, then yes they'll be changed after a power cycle or an `ipmi mc reset warm`. It's easy to put the ipmi commands needed into a startup script though if they turn out to be needed.

What does you `ipmitool sensor` output look like currently?

 

Do you want to see the output of sensor when in optimal mode or full speed?

 

Here is my sensor output during a period where the fans oscillate:

Code:

CPU Temp         | 24.000     | degrees C  | ok    | 0.000     | 0.000     | 0.000     | 95.000    | 98.000    | 100.000  
System Temp      | 23.000     | degrees C  | ok    | -9.000    | -7.000    | -5.000    | 80.000    | 85.000    | 90.000  
Peripheral Temp  | 31.000     | degrees C  | ok    | -9.000    | -7.000    | -5.000    | 80.000    | 85.000    | 90.000  
PCH Temp         | 43.000     | degrees C  | ok    | -11.000   | -8.000    | -5.000    | 90.000    | 95.000    | 100.000  
P1-DIMMA1 TEMP   | 29.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000  
P1-DIMMA2 TEMP   | 29.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000  
P1-DIMMB1 TEMP   | 28.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000  
P1-DIMMB2 TEMP   | 26.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000  
P1-DIMMC1 TEMP   | 25.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000  
P1-DIMMC2 TEMP   | 26.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000  
P1-DIMMD1 TEMP   | 27.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000  
P1-DIMMD2 TEMP   | 25.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000  
FAN 1            | 1275.000   | RPM        | ok    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
FAN 2            | 1200.000   | RPM        | ok    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
FAN 3            | 1800.000   | RPM        | ok    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
FAN 4            | 1950.000   | RPM        | ok    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
FAN A            | 1200.000   | RPM        | ok    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
Vcore            | 0.936      | Volts      | ok    | 0.480     | 0.512     | 0.544     | 1.488     | 1.520     | 1.552    
3.3VCC           | 3.328      | Volts      | ok    | 2.816     | 2.880     | 2.944     | 3.584     | 3.648     | 3.712    
12V              | 12.190     | Volts      | ok    | 10.494    | 10.600    | 10.706    | 13.091    | 13.197    | 13.303  
VDIMM            | 1.512      | Volts      | ok    | 1.152     | 1.216     | 1.280     | 1.760     | 1.776     | 1.792    
5VCC             | 5.056      | Volts      | ok    | 4.096     | 4.320     | 4.576     | 5.344     | 5.600     | 5.632    
CPU VTT          | 1.008      | Volts      | ok    | 0.872     | 0.896     | 0.920     | 1.344     | 1.368     | 1.392    
VBAT             | 3.488      | Volts      | ok    | 2.816     | 2.880     | 2.944     | 3.584     | 3.648     | 3.712    
VSB              | 3.584      | Volts      | ok    | 3.008     | 3.072     | 3.136     | 3.856     | 3.920     | 3.984    
AVCC             | 3.344      | Volts      | ok    | 2.816     | 2.880     | 2.944     | 3.584     | 3.648     | 3.712    
Chassis Intru    | 0x1        | discrete   | 0x0100| na        | na        | na        | na        | na        | na        

And here's the same while the fans are low RPM:

Code:

CPU Temp         | 23.000     | degrees C  | ok    | 0.000     | 0.000     | 0.000     | 95.000    | 98.000    | 100.000   
System Temp      | 23.000     | degrees C  | ok    | -9.000    | -7.000    | -5.000    | 80.000    | 85.000    | 90.000   
Peripheral Temp  | 31.000     | degrees C  | ok    | -9.000    | -7.000    | -5.000    | 80.000    | 85.000    | 90.000   
PCH Temp         | 43.000     | degrees C  | ok    | -11.000   | -8.000    | -5.000    | 90.000    | 95.000    | 100.000   
P1-DIMMA1 TEMP   | 29.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000   
P1-DIMMA2 TEMP   | 29.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000   
P1-DIMMB1 TEMP   | 28.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000   
P1-DIMMB2 TEMP   | 27.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000   
P1-DIMMC1 TEMP   | 25.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000   
P1-DIMMC2 TEMP   | 26.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000   
P1-DIMMD1 TEMP   | 27.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000   
P1-DIMMD2 TEMP   | 25.000     | degrees C  | ok    | 1.000     | 2.000     | 4.000     | 80.000    | 85.000    | 90.000   
FAN 1            | 375.000    | RPM        | ok    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
FAN 2            | 375.000    | RPM        | ok    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
FAN 3            | 300.000    | RPM        | nc    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
FAN 4            | 375.000    | RPM        | ok    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
FAN A            | 375.000    | RPM        | ok    | 150.000   | 225.000   | 300.000   | 18975.000 | 19050.000 | 19125.000
Vcore            | 0.864      | Volts      | ok    | 0.480     | 0.512     | 0.544     | 1.488     | 1.520     | 1.552     
3.3VCC           | 3.328      | Volts      | ok    | 2.816     | 2.880     | 2.944     | 3.584     | 3.648     | 3.712     
12V              | 12.190     | Volts      | ok    | 10.494    | 10.600    | 10.706    | 13.091    | 13.197    | 13.303   
VDIMM            | 1.504      | Volts      | ok    | 1.152     | 1.216     | 1.280     | 1.760     | 1.776     | 1.792     
5VCC             | 5.056      | Volts      | ok    | 4.096     | 4.320     | 4.576     | 5.344     | 5.600     | 5.632     
CPU VTT          | 1.008      | Volts      | ok    | 0.872     | 0.896     | 0.920     | 1.344     | 1.368     | 1.392     
VBAT             | 3.488      | Volts      | ok    | 2.816     | 2.880     | 2.944     | 3.584     | 3.648     | 3.712     
VSB              | 3.584      | Volts      | ok    | 3.008     | 3.072     | 3.136     | 3.856     | 3.920     | 3.984     
AVCC             | 3.344      | Volts      | ok    | 2.816     | 2.880     | 2.944     | 3.584     | 3.648     | 3.712     
Chassis Intru    | 0x1        | discrete   | 0x0100| na        | na        | na        | na        | na        | na       

FAN 1 - 120mm Noctua
FAN 2 - 120mm Noctua
FAN 3 - 80mm Noctua
FAN 4 - Noctua CPU cooler NH-D9DX i4 3U
FAN A - 120mm Noctua

 

Bugger, just noticed that nasty little NC on FAN 3 with low RPM.
I have lowered my lower thresholds for FAN 3, so NC is now 250. Seems ok now!

 

S'why I wanted you to post them, but glad you made the spot Some of your other low rpm's are a bit close to the assert thresholds as well so you might want to think about tweaking those downwards slightly too if you haven't already.

FYI, I noticed no issues* with setting the low thresholds to all the same value (e.g. 150 150 150) or just keeping the two lowest thresholds at zero with only having a positive asset (e.g. 0 0 150).


* Only in that it didn't ultimately work for me because my issue was the fans showing as 0rpm.

 

Hello!

I have the same issue. Did you finally fix it without any external controller?

Thanks by advance++

 

When you say the same issue, do you mean your fans spin up and down repeatedly?