Testing BIOS settings in LINUX, CPU frequencies, etc...

[DrStein99]

Ok, looks like in Windows this fancy operating system will automatically re-configure itself when I restart and enable, disable, and load it's own drivers for pretty much any BIOS setting I do. I've been spoiled by that, and was expecting Ubuntu server 16 to do the same thing. It does not.

I spent the last week trying to learn and do tests, getting really confused. There is an AHCI driver that loads or a P-STATE driver (or module) that loads when I install an Ubuntu 16 operating system. If I CHANGE my bios setting or GRUB, than the modules or drivers (or whatever needs to be done) is still set to however it was originally loaded when I installed the operating system. The TURBO cpu frequencies are also changed if I switch bios settings, since if I reduce the # of active cores, the maximum CPU frequency should change. I can't seem to trigger Linux to recognize I changed my bios settings and re-load the appropriate modules it needs to use.

I can't manage all this manually, and now have resorted to having to re-load my operating system from scratch every time I change a bios setting just to run another test.

Does anyone have advice on how I can do this easier ? Is there a procedure somewhere to trigger linux to rebuild it's own kernel, detect the CPU bios settings from the begninning again without me having to reload the operating system from scratch for each bios setting change?

 

[EffrafaxOfWug]

To put it bluntly, "what on earth are you talking about?"

I don't really understand what you mean when you say that the kernel isn't loading drivers or respecting your BIOS settings. Unless your hardware is continually changing, which I doubt, the OS and kernel should load the same drivers at every boot. You can see which drivers have been loaded using lsmod (at least those that are kernel modules).

There is an AHCI driver that loads or a P-STATE driver (or module) that loads when I install an Ubuntu 16 operating system. If I CHANGE my bios setting or GRUB, than the modules or drivers (or whatever needs to be done) is still set to however it was originally loaded when I installed the operating system.

There's no AHCI-specific driver in linux, TTBOMK this falls under sd_mod which automatically detects whether it's running on AHCI-enabled hardware or not. What modifications are you making that are not being re-applied on reboot?

Ditto for p-state; again that's typically auto-loaded when the OS boots. What is it you're changing about the p-state settings that isn't there after you reboot?

The TURBO cpu frequencies are also changed if I switch bios settings, since if I reduce the # of active cores, the maximum CPU frequency should change.

AFAICR that's entirely a function of the hardware and the OS has no say in how much the turbo clocks can be increased. It's possible under windows you've got a utility/driver provided by the motherboard vendor to tweak this value from within the OS. There's a utility available called turbostat that should give you a rundown of what clocks are available/utilised although I can't vouch for its accuracy. However a quick shufti at the kernel doc says that all the info in /proc is intentionally limited to the maximum reported clock speed of the CPU, so many clockspeed reporting utilities might not show boost clocks.

I assume that when you disable cores in ths BIOS, the change is reflected in the amount of CPUs visible under /proc/cpuinfo or lscpu?

I can't manage all this manually, and now have resorted to having to re-load my operating system from scratch every time I change a bios setting just to run another test.

You're reinstalling after changing a setting in the BIOS? Why?

 

[DrStein99]

To put it bluntly, "what on earth are you talking about?"

I apologize. I re-read my posting again after some rest after working on the issue for so long, it's clear that it has driven me a little crazy.

There's no AHCI-specific driver in linux, TTBOMK this falls under sd_mod which automatically detects whether it's running on AHCI-enabled hardware or not. What modifications are you making that are not being re-applied on reboot?

Dell r810 server. Under "processor settings", there are 2 settings. I am toggling them in the BIOS. I can not confirm or deny what is going on on the operating system side yet. These settings confuse me, since the bios f1-help is vague (at best) and my google-research is overwhelming.

1) C1E: enable/disable
2) C state: enable/disable

If I am not mistaken, these two settings affect the registers that allow the operating system to change the CPU frequency in whatever mode (that I do not yet fully understand). The only way I have to confirm what I have read about so far, is simply to change the settings and run my tests to see if/what/how any difference in performance.

----

There is another setting in bios, which should also affect the CPU frequency/turbo speed upper limit. I change this, and can not verify the speed-boost limit is changing in the operating system:

3) Number of core processor: 1/2/4/6/8/all

If I change the number of core processors available, reduce them from "all" to "6", I should be able to change that upper limit on the frequency. 2,133.33 base frequency for all 10 cores. 2,266.66 mhz for 6 cores. I know this does not seem like alot of speed. The server has 4 processors, so with all the cores and threads - the overall speed/turbo boost actually changes the performance of my application.

(This link is an example of reference for the information I am gathering. At the bottom of the page lists the cores and turbo frequency)
Xeon E7-4850 - Intel

I only have a use for 6 cores (per processor, 4 processors installed), due to the 24mb L3 cache available. I reduce the cores to 6, maximize the upper cpu-frequncy limit on the cores I am using.

I was trying to ask, how - if there was a way - for me to confirm the changes I make in the bios if I am to be loading a different module or something else, I do not know. I go into linux, check the cpu*/scaling frequencies and it kept showing the same frequencies no matter what I changed in the bios. Which lead me to believe that I had to toggle or something else extra in order to get the kernal to recognize the changes in core, c-states, or otherwise.

The turbostat only showed the core-frequency, no matter what I tried to change in the BIOS. So it seems I have just limited the number of cores available in the bios, change the settings and all I tested resulted in just slowing the machine down and reducing resources, instead of boosting the cpu speed - to the only cores I plan to use. I got desperate to find a solution, so toggling the C1E, and c-state appeared to by my other option to get some resolution.

 

[EffrafaxOfWug]

I apologize. I re-read my posting again after some rest after working on the issue for so long, it's clear that it has driven me a little crazy.

't wouldn't be STH if at least someone wasn't a little crazy!

1) C1E: enable/disable
2) C state: enable/disable

I'm not familiar with the Del R810, but those C-toggles are basically enabling/disabling the chip from entering certain C-states; these are part of the ACPI specs and govern what power-saving methodologies can be used. C1E is one of the basic power-saving methods used when the CPU isn't actively doing anything, and I suspect the second toggle covers all he other C-states the CPU supports. In most scenarios non-crazy people will want to leave these enabled as they use power/generate heat for (usually) very little appreciable benefit.

More modern CPUs support more and more complex C-states in order to save power. For example, here's powertop on my laptop showing my skylake going all the way to C10.

PowerTOP 2.8      Overview   Idle stats   Frequency stats   Device stats   Tunables                                     


          Package   |             Core    |            CPU 0       CPU 2
                    |                     | C0 active   2.6%        2.8%
                    |                     | POLL        0.0%    0.0 ms  0.0%    0.0 ms
                    |                     | C1E-SKL     0.7%    0.1 ms  0.7%    0.2 ms
C2 (pc2)   68.9%    |                     |
C3 (pc3)    0.0%    | C3 (cc3)    0.7%    | C3-SKL      1.0%    0.3 ms  0.9%    0.2 ms
C6 (pc6)    0.0%    | C6 (cc6)    7.9%    | C6-SKL      8.8%    0.8 ms  4.2%    0.5 ms
C7 (pc7)    0.0%    | C7 (cc7)   70.7%    | C7s-SKL     0.0%    0.1 ms  0.0%    0.7 ms
C8 (pc8)    0.0%    |                     | C8-SKL     26.0%    1.6 ms 20.3%    3.2 ms
C9 (pc9)    0.0%    |                     | C9-SKL      0.0%    0.0 ms  0.0%    0.0 ms
C10 (pc10)  0.0%    |                     | C10-SKL    51.1%   13.1 ms 61.8%   12.9 ms

                    |             Core    |            CPU 1       CPU 3
                    |                     | C0 active   2.9%        3.0%
                    |                     | POLL        0.0%    0.0 ms  0.0%    0.0 ms
                    |                     | C1E-SKL     0.9%    0.4 ms  1.4%    0.4 ms
                    |                     |
                    | C3 (cc3)    1.2%    | C3-SKL      1.3%    0.4 ms  1.1%    0.3 ms
                    | C6 (cc6)    2.2%    | C6-SKL      3.0%    0.4 ms  3.3%    0.5 ms
                    | C7 (cc7)   75.0%    | C7s-SKL     0.0%    0.0 ms  0.0%    0.2 ms
                    |                     | C8-SKL     14.0%    3.0 ms 19.8%    3.2 ms
                    |                     | C9-SKL      0.0%    0.5 ms  0.0%    0.0 ms
                    |                     | C10-SKL    67.9%   16.3 ms 62.3%   14.6 ms

                    |             GPU     |
                    |                     |
                    | Powered On  1.3%    |
                    | RC6        98.7%    |
                    | RC6p        0.0%    |
                    | RC6pp       0.0%    |
                    |                     |
                    |                     |
                    |                     |
                    |                     |
                    |                     |

If I am not mistaken, these two settings affect the registers that allow the operating system to change the CPU frequency in whatever mode (that I do not yet fully understand).

TTBOMK they only change the various power-saving methods used by the CPU and don't have any direct effect on frequency.

There is another setting in bios, which should also affect the CPU frequency/turbo speed upper limit. I change this, and can not verify the speed-boost limit is changing in the operating system:

3) Number of core processor: 1/2/4/6/8/all

If I change the number of core processors available, reduce them from "all" to "6", I should be able to change that upper limit on the frequency. 2,133.33 base frequency for all 10 cores. 2,266.66 mhz for 6 cores. I know this does not seem like alot of speed. The server has 4 processors, so with all the cores and threads - the overall speed/turbo boost actually changes the performance of my application.

CPUs don't generally work like that. This BIOS setting is basically there for people either limited to extremely tight power/heat budgets or stuck with running software that's licensed per CPU - so if you don't need an 8CPU oracle DB, you can turn it into a 2P server easily. This will have some decrease in thermals, but max clocks and the like will stay the same. Turbo boost might get you higher for slightly longer depending on the heat and the way max turbo is calculated, but any increase will be marginal at best.

(This link is an example of reference for the information I am gathering. At the bottom of the page lists the cores and turbo frequency)
Xeon E7-4850 - Intel

The max. frequency I see listed on that page is 2.4GHz. I'm not aware of the existence of any mechanism to increase this other than overclocking and, well, I doubt that's even possible on an E7.

I was trying to ask, how - if there was a way - for me to confirm the changes I make in the bios if I am to be loading a different module or something else, I do not know. I go into linux, check the cpu*/scaling frequencies and it kept showing the same frequencies no matter what I changed in the bios. Which lead me to believe that I had to toggle or something else extra in order to get the kernal to recognize the changes in core, c-states, or otherwise.

The turbostat only showed the core-frequency, no matter what I tried to change in the BIOS. So it seems I have just limited the number of cores available in the bios, change the settings and all I tested resulted in just slowing the machine down and reducing resources, instead of boosting the cpu speed - to the only cores I plan to use. I got desperate to find a solution, so toggling the C1E, and c-state appeared to by my other option to get some resolution.

Aye, I think you're under a misapprehension here. Neither changing the C-states nor disabling cores should have any effect at all on the maximum CPU speed, especially on a chip like an E7 and no amount of mucking about in /sys will change that I don't think.

 

[DrStein99]

The max. frequency I see listed on that page is 2.4GHz. I'm not aware of the existence of any mechanism to increase this other than overclocking and, well, I doubt that's even possible on an E7.

This is what I am trying to ultimately ask/solve here on the thread (to add some clairity on my haywire ramblings). I'm trying to test now as I write the message. So for example when I type:

sudo dmidecode -t processor | grep Speed

I get:

       Max Speed: 3600 MHz
        Current Speed: 2000 MHz
        Max Speed: 3600 MHz
        Current Speed: 2000 MHz
        Max Speed: 3600 MHz
        Current Speed: 2000 MHz
        Max Speed: 3600 MHz
        Current Speed: 2000 MHz

cat /proc/cpuinfo | grep "MHz"

returns:

cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000
cpu MHz         : 1996.000

I do a bunch other cat /proc's and cpu-freaky tools and pretty much they all return a value for processor speed of 1996000. I do not know if 1,996,000 is the absolute highest this will ever go or what I need to do to enable it to go above 2,000,000.

I wish my notes were not scattered all over the place, I could give you what I have done so far, but I can not make sense of it myself. I promise you, I have been reading ubuntu guides on changing the frequency scale, maximum and have done so many I lost track.

I know something is wrong I am certainly doing. On this other machine core i5 2400, the max cpu on the other machine I am trying to compare to.

• 3.10 GHz
• Max Turbo Frequency3.40 GHz

I go on the core i5 machine do:

cat /proc/cpuinfo | grep "MHz"

and IT will return:

cpu MHz         : 3199.902
cpu MHz         : 3299.925
cpu MHz         : 3299.925
cpu MHz         : 3299.320

So this was the difference that confuses me. The core i-5 machine reports clock speed into the turbo range. So whatever magic setting I changed on that machine I must have forgot. I think all I did was issue "performance mode" to the scaling governer (but my memory is poor).
I feel like I tried to change everything, but nothing works to get the cpu mhz to change on the e7.

Here is another interesting piece of information on the e7:

cat/sys/devices/system/cpu/cpu0/cpufreq/bios_limit
1996000

on the core i5 system:

 cat /sys/devices/system/cpu/cpu0/cpufreq/bios_limit
cat: /sys/devices/system/cpu/cpu0/cpufreq/bios_limit: No such file or directory

 

[EffrafaxOfWug]

I've never seen the bios_limit in cpufreq before but on the face of it suggests that the BIOS is telling linux not to ever clock above 2GHz for whatever reason. You can force this to be overridden by writing 1 to ignore_ppc apparently;

echo 1 > /sys/module/processor/parameters/ignore_ppc

...and then checking to see if the maximum listed frequency from

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq

is attainable.

But bear in mind you're going past the automatically agreed upon values here so I'd be wary about thermal load and power supply stability, as per the warning at the top of this page.

P.S. an easy way to watch the current core speeds should be:

watch -n0.5 "cat /proc/cpuinfo | grep MHz"

 

[MBastian]

I'd suggest you reset the BIOS to default, then (temporary) disable all C-States and set all Power Management settings to "Maximum Performance". Also confirm that the installed power supply is up to the task. See the owners manual for details.

Instead of sifting through /proc ans /sys I'd reccoment such utils as cpupower ,turbostat and perhaps for confirmation cpufreq-bench.

Default on my power guzzling dual E5-2670 workstation

[root@colossus ~]# cpupower  frequency-info
analyzing CPU 0:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency:  Cannot determine or is not supported.
  hardware limits: 1.20 GHz - 3.30 GHz
  available cpufreq governors: performance powersave
  current policy: frequency should be within 1.20 GHz and 3.30 GHz.
                  The governor "powersave" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 2.59 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes
    3200 MHz max turbo 4 active cores
    3200 MHz max turbo 3 active cores
    3300 MHz max turbo 2 active cores
    3300 MHz max turbo 1 active cores

To change the governor to "performance"

cpupower frequency-set -g performance

To get even more information have a look at ipmitool It's a very powerful tool that can dump you a treasure throve of information. Eg. ipmitool sel elist, ipmitool sensor and ipmitool chassis status.
If your server does not have ipmi you can use lm-sensors to get to the temperature and fan RPMs.

 

[DrStein99]

I'd suggest you reset the BIOS to default, then (temporary) disable all C-States and set all Power Management settings to "Maximum Performance". Also confirm that the installed power supply is up to the task. See the owners manual for details.

Ok thanks. I can update the bios settings, no problem. I was confused after I update the bios settings - if I need to load modules or drivers in linux/ubuntu on those changes.

I will just load Windows 10 on this machine, so I can have something to compare results with.