Skip to contentSkip to conversation

Disable cpufreq on modern X86 processors

DISCUSSION Thomas Renninger 4 participants 5 messages Jul 24, 2012 – Jul 26, 2012

lore
Share & tools
lore
#0 lore raw

Hi,

I recently got pointed to performance losses measured with and without cpufreq enabled when people worked on scheduler tunables/improvements.

Depending on whether processes are bound to cores, tunables inside the cpufreq subsystem, etc. there can be rather big differences.

While there have been improvements (for example do not poll that often if constantly running at highest frequency and others), dynamic cpufreq adjusting as it currently is implemented via ondemand/conservative governors always will cost performance.

Arjan mentioned quite some time ago, that for modern X86 processors it does not make much sense to control the frequency of the CPU via OS, because idle states are much more efficient and should get entered asap.

Especially on bigger X86 systems with dozens or even hundreds of cores, cpufreq polling sounds like a bad idea. Especially if the CPUs do achieve the same or even better performance/power results via entering C-states quickly.

I would like to come up with a init_default_governor() or similar function which choses the performance governor for such CPUs.
Hm, maybe it could get a driver callback, then this one could be picked up by acpi-cpufreq (and powernow-k8 if applicable) and those drivers could choose the right governor for the platform/cpu.

Ideally identifying the CPUs where performance governor should get used is a one liner checking for a cpu flag.
But this might not get that easy? CPU family/model would need maintenance if there is no cpu flag/feature to test for.

Just some ideas..., if it's doable with some lines of code without the need of maintaining/adding new cpu families, I'd like to have a better default behavior.

One main problem I am facing is: Measuring power consumption in different workloads.

I can measure the power consumption in idle (deeper sleep states entered) when CPU frequency is set to lowest and highest and compare. If both are the same, the CPU is a good candidate to not do OS controlled CPU frequency scaling.

What do you think?

Thanks,

Thomas

#1 lore raw

CC-ing Andre

Quoted 57 lines · Thomas Renninger

Hi,

I recently got pointed to performance losses measured with and without cpufreq enabled when people worked on scheduler tunables/improvements.

Depending on whether processes are bound to cores, tunables inside the cpufreq subsystem, etc. there can be rather big differences.

While there have been improvements (for example do not poll that often if constantly running at highest frequency and others), dynamic cpufreq adjusting as it currently is implemented via ondemand/conservative governors always will cost performance.

Arjan mentioned quite some time ago, that for modern X86 processors it does not make much sense to control the frequency of the CPU via OS, because idle states are much more efficient and should get entered asap.

Especially on bigger X86 systems with dozens or even hundreds of cores, cpufreq polling sounds like a bad idea. Especially if the CPUs do achieve the same or even better performance/power results via entering C-states quickly.

I would like to come up with a init_default_governor() or similar function which choses the performance governor for such CPUs.
Hm, maybe it could get a driver callback, then this one could be picked up by acpi-cpufreq (and powernow-k8 if applicable) and those drivers could choose the right governor for the platform/cpu.

Ideally identifying the CPUs where performance governor should get used is a one liner checking for a cpu flag.
But this might not get that easy? CPU family/model would need maintenance if there is no cpu flag/feature to test for.

Just some ideas..., if it's doable with some lines of code without the need of maintaining/adding new cpu families, I'd like to have a better default behavior.

One main problem I am facing is: Measuring power consumption in different workloads.

I can measure the power consumption in idle (deeper sleep states entered) when CPU frequency is set to lowest and highest and compare. If both are the same, the CPU is a good candidate to not do OS controlled CPU frequency scaling.

What do you think?

Thanks,

Thomas

#2 lore raw
Quoted 36 lines · Andreas Herrmann

CC-ing Andre

Quoted 33 lines · Thomas Renninger

Hi,

I recently got pointed to performance losses measured with and without cpufreq enabled when people worked on scheduler tunables/improvements.

Depending on whether processes are bound to cores, tunables inside the cpufreq subsystem, etc. there can be rather big differences.

While there have been improvements (for example do not poll that often if constantly running at highest frequency and others), dynamic cpufreq adjusting as it currently is implemented via ondemand/conservative governors always will cost performance.

Arjan mentioned quite some time ago, that for modern X86 processors it does not make much sense to control the frequency of the CPU via OS, because idle states are much more efficient and should get entered asap.

Especially on bigger X86 systems with dozens or even hundreds of cores, cpufreq polling sounds like a bad idea. Especially if the CPUs do achieve the same or even better performance/power results via entering C-states quickly.

I would like to come up with a init_default_governor() or similar function which choses the performance governor for such CPUs.
Hm, maybe it could get a driver callback, then this one could be picked up by acpi-cpufreq (and powernow-k8 if applicable) and those drivers could choose the right governor for the platform/cpu.

Actually we are currently also looking into this issue.
So I'd refrain at least for now from changing the default governor.
Let's see what Arjan comes up with, I'd also like to see a reworked governor instead of changing the default one.

To answer your questions nevertheless:

Quoted 4 lines
Quoted 4 lines

Ideally identifying the CPUs where performance governor should get used is a one liner checking for a cpu flag.
But this might not get that easy? CPU family/model would need maintenance if there is no cpu flag/feature to test for.

From the AMD side it looks like family >= 0x11 should do the trick. The actual difference is the availability of >C1 per core sleep states. Family 10h and 11h have C1 and C1e, but no real deeper states. Let me see if there is some register or bit we can reliably query to check for this instead of relying on fragile f/m/s values.

On Phenoms I could measure much lower power usage with ondemand (or conservative) governor on idle. This was not true for Bulldozers anymore, so there is some truth in your idea.

Regards,
Andre.

Quoted 19 lines
Quoted 19 lines

Just some ideas..., if it's doable with some lines of code without the need of maintaining/adding new cpu families, I'd like to have a better default behavior.

One main problem I am facing is: Measuring power consumption in different workloads.

I can measure the power consumption in idle (deeper sleep states entered) when CPU frequency is set to lowest and highest and compare. If both are the same, the CPU is a good candidate to not do OS controlled CPU frequency scaling.

What do you think?

Thanks,

Thomas

Signature · 3 lines
Andre Przywara
AMD-OSRC (Dresden)
Tel: x29712
#3 lore raw
Quoted 3 lines · Andre Przywara

On Phenoms I could measure much lower power usage with ondemand (or conservative) governor on idle. This was not true for Bulldozers anymore, so there is some truth in your idea.

for Intel, the very hard split came with Nehalem for real, although C1E earlier also changed the rules already.

ondemand is 10 years old, and the rules of the hardware have changed fundamentally twice since.

(and frankly, and this may not be very popular on this list, CPUFREQ is pretty much beyond fixing. The locking rules are completely hosed, but more, the assumption that you can or should detangle policy from the hardware is fundamentally flawed. The policy IS hardware specific. Because it's tuned for specific hardware behavior, if you actually optimize it)

#4 lore raw
In reply to Thomas Renninger
Quoted 5 lines

Arjan mentioned quite some time ago, that for modern X86 processors it does not make much sense to control the frequency of the CPU via OS, because idle states are much more efficient and should get entered asap.

well need to qualify that.
it does not make sense to control the frequency the way we are doing, because during idle the frequency is zero (as is the voltage). ondemand logic has different assumptions.

we have been reworking a new governer like thing inside Intel, and are close to publish it to lkml (and this list).
the problem always is that there are many workloads and one shouldn't regress bla bla bla so this is needs a lot of data collection (which takes time)