Understanding cpufreq?
I have a number of dual CPU and dual CPU/dual core Opteron systems that are used as compute servers. In an effort to reduce power consumption and reduce heat output, I would like to make use of the PowerNow! capabilities to clock back the CPUs when the machines are idle.
These machines are running a 2.6.9-42 RHEL4 kernel with the powernow-k8 module loaded - which I believe have backported cpufreq support from more recent mainline kernels.
In trying to achieve what I want, I've become rather confused as to how cpufreq in a multi-CPU environment works:
There is a directory under /sys/devices/system/cpu/cpu*/cpufreq for each CPU, which seems to imply that each CPU speed can be controlled separately - can this really be the case? Can separate CPU cores run at different speeds?
e.g. I can echo 4 different governor names to the scaling_governor file in each /sys/devices/system/cpu/cpu[0-3]/cpufreq directory on a 4 core machine - and the resulting scaling_cur_freq file can contain a different value.
However, the "cpu MHz" fields in /proc/cpuinfo are all the same for each each CPU - I assume the values in /proc/cpuinfo are the 'correct' values ??
Also, if I set all the governors to userspace, and then set each CPU's speed via scaling_setspeed to a different (allowed) value, then it appears quite random as to which value is then reflected in /proc/cpuinfo i.e. sometimes it will take the value given to CPU 0, other times it will be CPU 1 etc.
If I set all the governors to ondemand, the CPUs will from time to time, clock back their speed in situations where one or more CPUs are being heavily used. i.e it appears that each CPU is treated separately, and if one CPU is deemed to be idle enough by its given metrics, then it can reduce the speed of all CPUs, regardless of other CPUs being 'busy' ...
I've also tried a couple of userspace daemons (cpuspeed and powernowd) - again, these treat each CPU separately and will also reduce the speed of an 'idle' CPU - and hence reduce the speed of all the CPUs, again, regardless of other CPUs being 'busy'.
Essentially what I want to achieve is something like: if _any_ CPU is 'busy' (usage over some threshold over some sampling period), then run at full speed and if _all_ CPUs are 'idle' (all below some threshold over some sampling period) then clock back the CPUs.
Is there something/some setting(s) that can do this in a multi-CPU machine?
Thanks
James Pearson
Quoted 3 lines
These machines are running a 2.6.9-42 RHEL4 kernel with the powernow-k8 module loaded - which I believe have backported cpufreq support from more recent mainline kernels.
To an extent. The problem is not the powernow-k8 driver, but the cpufreq infrastructure. See below.
Quoted 8 lines
In trying to achieve what I want, I've become rather confused as to how cpufreq in a multi-CPU environment works:
There is a directory under
/sys/devices/system/cpu/cpu*/cpufreqfor each
CPU, which seems to imply that each CPU speed can be controlled separately - can this really be the case? Can separate CPU cores run at different speeds?
No, but prior to 2.6.10, cpufreq discovers each core independently, and creates a directory independently. The powernow-k8 knows which cores are on the same processors, and handles them together.
After 2.6.10, cpufreq understands multicore devices and creates 1 directory for the entire processor and then creates symbolic links for all the cores that share the same pstates.
Quoted 10 lines
e.g. I can echo 4 different governor names to the
scaling_governorfile in each /sys/devices/system/cpu/cpu[0-3]/cpufreq directory on a 4 core machine - and the resultingscaling_cur_freqfile can contain a different value.However, the "cpu MHz" fields in
/proc/cpuinfoare all the same for each each CPU - I assume the values in/proc/cpuinfoare the 'correct' values ??
Quoted 6 lines
Also, if I set all the governors to userspace, and then set each CPU's speed via
scaling_setspeedto a different (allowed) value, then it appears quite random as to which value is then reflected in/proc/cpuinfoi.e. sometimes it will take the value given to CPU 0,
other times it will be CPU 1 etc.
It's always the highest requested value of all cores on a processor. powernow-k8 keeps track of the requested frequencies, though, so you can have some weird situations.
ie, from boot up:
core0 800 MHz core1 800 MHz CPU 800 MHz core0 to 2000 MHz core1 800 MHz CPU 2000 MHz core0 2000 MHz core1 to 1800 MHz CPU 2000 MHz core0 to 800 MHz core1 1800 MHz CPU 1800 MHz
Quoted 7 lines
If I set all the governors to ondemand, the CPUs will from time to time, clock back their speed in situations where one or more CPUs are being heavily used. i.e it appears that each CPU is treated separately, and if
one CPU is deemed to be idle enough by its given metrics, then it can reduce the speed of all CPUs, regardless of other CPUs being 'busy' ...
That shouldn't happen, unless ondemand requests a low frequency to all cores.
Quoted 5 lines
Essentially what I want to achieve is something like:
if _any_ CPU is 'busy' (usage over some threshold over some sampling period), then run at full speed and if _all_ CPUs are 'idle' (all below some threshold over some sampling period) then clock back the CPUs.
This should be happening automatically in RHEL4.
Note that the 2.6.10 kernel changed cpufreq behavior dramatically, and now the most recent frequency request wins.
-Mark Langsdorf
powernow-k8 maintainer
AMD, Inc.