Q6600 CPU Frequency Scaling Problem / Bug

Hi,

I'm running Debian testing on an Intel Core2Quad Q6600 with 4GB DDR2 RAM on a MSI P35 Platinum board since December 07. When I first migrated my system on December (I've installed Debian as etch beta1 and upgrading since), every core of the CPU was scaling their frequency independently. After installing 2.6.22, The cores started to scale in a synchronized way. I.e. when a core needs to speed-up, every core speeds up in sync. The weird thing is, I have another clone of this OS at my office running on a core 2 duo processor and that system scaled its processors independently.

The ganged scaling behavior is also consistent with the /sys/devices/system/cpuX/cpufreq/affected_cpus file and cpufreq-info command. Both of them shows that all CPU's speeds are need to be in sync while there's really no need. Also Ubuntu 8.04 Live CD exhibits the same behavior and this behavior can be verified under sysfs again.

Is it a bug? Is it expected? I can provide more information if it's needed. I'm attaching cpufreq-info's output as a reference.

Cheers and Regards,

-- Hakan.

cpufrequtils 002: cpufreq-info (C) Dominik Brodowski 2004-2006
Report errors and bugs to [email protected], please.
analyzing CPU 0:
driver: acpi-cpufreq
CPUs which need to switch frequency at the same time: 0 1 2 3
hardware limits: 1.60 GHz - 2.40 GHz
available frequency steps: 2.40 GHz, 2.14 GHz, 1.87 GHz, 1.60 GHz available cpufreq governors: userspace, powersave, ondemand, conservative, performance current policy: frequency should be within 1.60 GHz and 2.40 GHz.
The governor "ondemand" may decide which speed to use within this range.
current CPU frequency is 1.60 GHz.
analyzing CPU 1:
driver: acpi-cpufreq
CPUs which need to switch frequency at the same time: 0 1 2 3
hardware limits: 1.60 GHz - 2.40 GHz
available frequency steps: 2.40 GHz, 2.14 GHz, 1.87 GHz, 1.60 GHz available cpufreq governors: userspace, powersave, ondemand, conservative, performance current policy: frequency should be within 1.60 GHz and 2.40 GHz.
The governor "ondemand" may decide which speed to use within this range.
current CPU frequency is 1.60 GHz.
analyzing CPU 2:
driver: acpi-cpufreq
CPUs which need to switch frequency at the same time: 0 1 2 3
hardware limits: 1.60 GHz - 2.40 GHz
available frequency steps: 2.40 GHz, 2.14 GHz, 1.87 GHz, 1.60 GHz available cpufreq governors: userspace, powersave, ondemand, conservative, performance current policy: frequency should be within 1.60 GHz and 2.40 GHz.
The governor "ondemand" may decide which speed to use within this range.
current CPU frequency is 1.60 GHz.
analyzing CPU 3:
driver: acpi-cpufreq
CPUs which need to switch frequency at the same time: 0 1 2 3
hardware limits: 1.60 GHz - 2.40 GHz
available frequency steps: 2.40 GHz, 2.14 GHz, 1.87 GHz, 1.60 GHz available cpufreq governors: userspace, powersave, ondemand, conservative, performance current policy: frequency should be within 1.60 GHz and 2.40 GHz.
The governor "ondemand" may decide which speed to use within this range.
current CPU frequency is 1.60 GHz.

_______________________________________________
Cpufreq mailing list
[email protected]
http://lists.linux.org.uk/mailman/listinfo/cpufreq

Just to confirm what I understood:
- Older kernels, each logical CPU can show different frequencies and affected_cpus has only once CPU number in it.
- Newer kernel, logical CPUs show same freq and affected_cpus includes all CPUs in the platform.

Correct?
In this case it seems to be the expected behavior.

In actual hardware, voltage is coordinated at socket level and that is the reason frequencies in one socket are tied together. Now, what has changed in two above config will be the mode in which kernel operates:
1) Hardware coordination mode: Kernel thinks each core is having independent frequency and reports the same. Underneath, hardware does frequency coodination and picks the highest requested frequency among all cores and runs all cores at that freq.
2) Software coordination mode: Kernel understands which specific CPUs are dependent and picks the highest frequency needed among all such dependent cores and makes single request for such frequency and reports the same.

Note that there should not be any power consumption difference with these two kernels on under identical load. Just that the kernel now knows more about the frequency dependencies in the platform.

Thanks,
Venki

Hi again,

No this is not the case because, none of my CPUs are logical. I have one true quad core and I true dual core CPU package. Each cores in these cpus are real and complete (not hyper threading CPUs). When I was running 2.6.21, both quad core and dual core systems were scaling its speeds independently per core. Currently, while my dual core system is continuing to do it, my quad core cpu is not doing it. Instead, it syncs its speed.

In reality, Core2Quad Q6600 is two Core2Duo E6600s. As A result I'm running a 2xDual Core CPUs in one package which each dual core CPU can scale every of its core independently from each other and when I run a single threaded intensive job, all of my cores scale up and I can see its effects in the output of sensors command. If the cores were logical or not-complete, Syncing should be the thing to do but When I'm running two of my office PC's CPUs (literally) in one package, it's hard to understand this behavior.

Thanks for reply,

Cheers and Regards,

--Hakan.

_______________________________________________
Cpufreq mailing list
[email protected]
http://lists.linux.org.uk/mailman/listinfo/cpufreq

________________________________
From: Hakan BAYINDIR [mailto:[email protected]]
Sent: Monday, June 30, 2008 1:33 PM
To: Pallipadi, Venkatesh
Cc: [email protected]
Subject: Re: Q6600 CPU Frequency Scaling Problem / Bug

Pallipadi, Venkatesh wrote:

-----Original Message-----
From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Hakan BAYINDIR
Sent: Monday, June 30, 2008 12:51 PM
To: [email protected]<mailto:[email protected]>
Subject: Q6600 CPU Frequency Scaling Problem / Bug

Hi,

I'm running Debian testing on an Intel Core2Quad Q6600 with 4GB DDR2 RAM
on a MSI P35 Platinum board since December 07. When I first migrated my system on December (I've installed Debian as etch beta1 and upgrading since), every core of the CPU was scaling their frequency independently.
After installing 2.6.22, The cores started to scale in a synchronized way. I.e. when a core needs to speed-up, every core speeds up in sync. The weird thing is, I have another clone of this OS at my office running
on a core 2 duo processor and that system scaled its processors independently.

The ganged scaling behavior is also consistent with the /sys/devices/system/cpuX/cpufreq/affected_cpus file and cpufreq-info command. Both of them shows that all CPU's speeds are need to be in sync
while there's really no need. Also Ubuntu 8.04 Live CD exhibits the same
behavior and this behavior can be verified under sysfs again.

Is it a bug? Is it expected? I can provide more information if it's needed. I'm attaching cpufreq-info's output as a reference.

Just to confirm what I understood:
- Older kernels, each logical CPU can show different frequencies and affected_cpus has only once CPU number in it.
- Newer kernel, logical CPUs show same freq and affected_cpus includes all CPUs in the platform.

Correct?
In this case it seems to be the expected behavior.

In actual hardware, voltage is coordinated at socket level and that is the reason frequencies in one socket are tied together. Now, what has changed in two above config will be the mode in which kernel operates:
1) Hardware coordination mode: Kernel thinks each core is having independent frequency and reports the same. Underneath, hardware does frequency coodination and picks the highest requested frequency among all cores and runs all cores at that freq.
2) Software coordination mode: Kernel understands which specific CPUs are dependent and picks the highest frequency needed among all such dependent cores and makes single request for such frequency and reports the same.

Note that there should not be any power consumption difference with these two kernels on under identical load. Just that the kernel now knows more about the frequency dependencies in the platform.

Thanks,
Venki

Hi again,

No this is not the case because, none of my CPUs are logical. I have one true quad core and I true dual core CPU package. Each cores in these cpus are real and complete (not hyper threading CPUs). When I was running 2.6.21, both quad core and dual core systems were scaling its speeds independently per core. Currently, while my dual core system is continuing to do it, my quad core cpu is not doing it. Instead, it syncs its speed.

In reality, Core2Quad Q6600 is two Core2Duo E6600s. As A result I'm running a 2xDual Core CPUs in one package which each dual core CPU can scale every of its core independently from each other and when I run a single threaded intensive job, all of my cores scale up and I can see its effects in the output of sensors command. If the cores were logical or not-complete, Syncing should be the thing to do but When I'm running two of my office PC's CPUs (literally) in one package, it's hard to understand this behavior.

Sorry for not being clear about logical CPU part. I meant CPUs sharing the same socket as opposed to multi-socket system and I did not mean Hyper-threading CPUs.

Both Core 2 Duo and Core 2 Quad, voltage is sync'ed across all cores in a single socket due to VR restriction. Most of the power savings from lower freq comes from lower voltage. As all cores in a single socket runs on same voltage here, independent voltage is not possible. On a real multi-socket system (dual or quad socket serves, cores in each socket can be at different frequencies though).

Older linux kernel only supports hardware coordination which explains the pre 2.6.21 behavior. Newer Linux kernel picks up hardware coordination mode or software coordination mode based on depending on BIOS capability and information it gets from BIOS ACPI table. So, it is possible that different systems have different coordination mode active, with same kernel.

Thanks,
Venki

Uh oh. I get it. So the long story short, "since we cannot vary voltage across the cores due to hardware design, we sync the cores, get more power without wasting too much power anyway". Am I right?

While I'm familiar with cpu architectures, I'm not familiar how Linux decides to use its features :).

Thanks for enlightenment and sorry for the fuss.

Cheers and Regards,

--Hakan

_______________________________________________
Cpufreq mailing list
[email protected]
http://lists.linux.org.uk/mailman/listinfo/cpufreq