Skip to contentSkip to conversation

related_cpus truncated with acpi-cpufreq driver on kernel 3.9.3

Bug 58761 [email protected] 1 participant 18 messages May 29, 2013 – Jun 19, 2013 1 message not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

Aaron Lu <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |ASSIGNED
                 CC|                            |[email protected]
          Component|Config-Processors           |cpufreq
         AssignedTo|acpi_config-processors@kern |[email protected]
                   |el-bugs.osdl.org            |
            Product|ACPI                        |Power Management
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#2 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

Viresh Kumar <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #1 from Viresh Kumar <[email protected]> 2013-05-29 06:07:45 --- (In reply to comment #0)

Quoted 6 lines

Since then, the related_cpus file in /sys/devices/system/cpu/cpu*/cpufreq/related_cpus just returns the id of the current core:

/sys/devices/system/cpu/cpu0/cpufreq/related_cpus contains 0 /sys/devices/system/cpu/cpu6/cpufreq/related_cpus contains 6 etc

Yes this is changed recently in kernel...
Now, affected cpus contain: All online cpus that share clock line related_cpus contain: All online+offline cpus that share clock line

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#3 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #2 from Jean-Philippe Halimi <[email protected]> 2013-05-29 15:26:03 --- @Viresh Kumar: Alright, I get this. But my Intel(R) Core(TM) i7-3770 CPU @
3.40GHz (Ivy Bridge) cpu does have threads that share clock frequency domain.

Here are the results any older version (from 3.2.0 at least) of the kernel have :
/sys/devices/system/cpu/cpu0/cpufreq/related_cpus: 0 1 2 3 4 5 6 7

Which means : Any thread within this set of threads physically apply the same frequency.

In the 3.9.3 kernel version, I don't have this result anymore (see my previous post), and I doubt the Linux kernel has changed any physical property of my processor.

According to what you said in your post, is "shared clock line" different from "shared frequency domain" ?

Thank you for your time

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#4 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

Dirk Brandewie <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #3 from Dirk Brandewie <[email protected]> 2013-05-29 15:48:40 --- The frequency/p-state is separate per thread, all threads non-idle threads will run at the frequency that will satisfy all the requests. The CPU itself chooses the actual P-state based on all the requests.

So all cores/threads are logically separate but do affect other cores.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#6 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #5 from Dirk Brandewie <[email protected]> 2013-05-29 16:09:49 --- So reflecting this relation up through sysfs is not particularly useful IMHO.

Showing only the logical connection is more correct IMHO.

I am not entirely understand why the change was made (not required :-) but is more accurate from my point of view.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#7 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #6 from Viresh Kumar <[email protected]> 2013-05-30 15:05:19 --- (In reply to comment #5)

Quoted 2 lines

I am not entirely understand why the change was made (not required :-) but is more accurate from my point of view.

I didn't understood what was the use of related_cpus in kernel at all earlier.. Sent many mails across lists and didn't got a satisfactory answer.. Searched kernel source and nobody seemed to be using it (including cpufreq core)..

And when I cleaned up the core finally it made more sense to use it in a separate way: i.e. have both online and offline cpus in it.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#8 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #7 from Jean-Philippe Halimi <[email protected]> 2013-05-30 15:09:05 --- @Dirk Brandewie Having the physical connection allows to understand how the
underlying hardware works. Having a file in cpuN that contains N only is
useless.

@Viresh Kumar Yes but on my Intel Core i7 processor, all the offline+online cores are sharing the same frequency domain, which means they should all appear in the related_cpu files.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#9 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #8 from Viresh Kumar <[email protected]> 2013-05-30 15:14:16 --- (In reply to comment #7)

Quoted 3 lines

@Viresh Kumar Yes but on my Intel Core i7 processor, all the offline+online cores are sharing the same frequency domain, which means they should all appear in the related_cpu files.

cpufreq doesn't care how actual hardware clock domains are managed, it just trusts whatever underlying cpufreq driver has communicated. Because x86 drivers want cpufreq core to believe that every core has a separate clock, so it is.

It doesn't make any sense what so ever to keep only one cpu in affected_cpus and all cpus 0-7 in related_cpus as that information isn't used by core. related cpus comes same as affected cpus in your case because you only have one core per domain (virtual domain :) ).. But in case you have more cores in a cluster and few of them are offlined, these two will have different values.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#10 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #9 from Jean-Philippe Halimi <[email protected]> 2013-05-30 15:44:20 ---

Quoted 5 lines

It doesn't make any sense what so ever to keep only one cpu in affected_cpus and all cpus 0-7 in related_cpus as that information isn't used by core. related cpus comes same as affected cpus in your case because you only have one core per domain (virtual domain :) ).. But in case you have more cores in a cluster and few of them are offlined, these two will have different values.

I am not arguing for or against the difference between affected_cpus and related_cpus. For me so far, the difference between the two was:
- affected_cpus was a "list of CPUs that require software coordination of frequency"
- related_cpus was a "List of CPUs that need some sort of frequency coordination, whether software or hardware."

This is at least what the official cpufreq Kernel documentation states (see
https://www.kernel.org/doc/Documentation/cpu-freq/user-guide.txt)

There definitely is hardware coordination between the 8 cores of my machine. Plus we can easily think about future architectures where a single processor can have distinct frequency domains. So the "frequency domain" notion is not a trivial question (all the cores of a single processor, or some cores of the processor), and has been an information Linux has been giving in previous releases of the kernel (before 3.9). If I get it right, your point is that losing this information is "more logical", and I am saying that losing this information is a loss of information. :)

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#11 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #10 from Viresh Kumar <[email protected]> 2013-05-30 15:53:55 --- (In reply to comment #9)

Quoted 2 lines

I am not arguing for or against the difference between affected_cpus and related_cpus.

I understand.

Quoted 8 lines

For me so far, the difference between the two was:
- affected_cpus was a "list of CPUs that require software coordination of frequency"
- related_cpus was a "List of CPUs that need some sort of frequency coordination, whether software or hardware."

This is at least what the official cpufreq Kernel documentation states (see
https://www.kernel.org/doc/Documentation/cpu-freq/user-guide.txt)

You need to check the latest ones:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/cpu-freq/user-guide.txt?id=refs/tags/v3.9

Quoted 8 lines

There definitely is hardware coordination between the 8 cores of my machine. Plus we can easily think about future architectures where a single processor can have distinct frequency domains. So the "frequency domain" notion is not a trivial question (all the cores of a single processor, or some cores of the processor), and has been an information Linux has been giving in previous releases of the kernel (before 3.9). If I get it right, your point is that losing this information is "more logical", and I am saying that losing this information is a loss of information. :)

Why I removed it was: "Nobody is making use of this information and was turning out to be more misleading".. As "hardware" for kernel is whatever is present below kernel layer.. So, if the layers below linux don't want to hide that information, let it be.

But yes, if we have broken something with that change it must be fixed. But that is not the case. It is just that you saw different values in it with latest kernels and got worried about if something went wrong.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#12 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #11 from Jean-Philippe Halimi <[email protected]> 2013-05-30 16:00:09 ---

Quoted 3 lines

You need to check the latest ones:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/cpu-freq/user-guide.txt?id=refs/tags/v3.9

Thanks for the link, I didn't know the other one was outdated

Quoted 4 lines

Why I removed it was: "Nobody is making use of this information and was turning out to be more misleading".. As "hardware" for kernel is whatever is present below kernel layer.. So, if the layers below linux don't want to hide that information, let it be.

I have actually been using these files for while for a project related with energy in order to decide which cores have a physical correlation with respect to frequency transition. So if I get it right, it is not possible to have such information anymore.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#13 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

Viresh Kumar <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #12 from Viresh Kumar <[email protected]> 2013-05-30 16:05:03 --- (In reply to comment #11)

Quoted 9 lines
Quoted 4 lines

Why I removed it was: "Nobody is making use of this information and was turning out to be more misleading".. As "hardware" for kernel is whatever is present below kernel layer.. So, if the layers below linux don't want to hide that information, let it be.

I have actually been using these files for while for a project related with energy in order to decide which cores have a physical correlation with respect to frequency transition. So if I get it right, it is not possible to have such information anymore.

Yeah.. I am not sure if it is worth to add another field (only for acpi-cpufreq) to get this information back..

@Rafael: ??

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#14 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #13 from Dirk Brandewie <[email protected]> 2013-05-30 16:28:26 --- Finding the correlation between cores is a LOT more complex than looking at the information that was presented by related_cpus. The correlation is workload dependant idle cores lose their vote for what the package p-state should be.

IMHO the information presented by related_cpus has not been useful since Nehalem came out.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#15 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

Benoit Pradelle <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #14 from Benoit Pradelle <[email protected]> 2013-05-30 16:51:31 --- I totally agree with Jean-Philippe: some information is lost with the 3.9 version and this information *is* useful to correctly set CPU frequencies. So either the related_cpus field meaning should be restored or a new field should be added.

If you doubt that such information is valuable, please consider all the runtime DVFS controllers such as beta adaptive [1], or CPU MISER [2]. Those systems, to be ported on current multicore processors, have to take a great care of the actually applied CPU frequency (not only that requested per core but the one actually in use). If it is impossible to determine what cores share the same frequency, it is impossible to perform such kind of DVFS on multicore CPUs in a portable way.

If you think that related_cpus is misleading (and I'd totally agree with you on that), then maybe a new field is required.

[1] C.-h. Hsu and W.-c. Feng, “A power-aware run-time system for high-performance computing,” in Proceedings of the 2005 ACM/IEEE conference on Supercomputing.
[2] R. Ge, X. Feng, W. chun Feng, and K. Cameron, “CPU MISER: A performance-directed, run-time system for power-aware clusters,” in Parallel Processing, 2007. ICPP 2007.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#16 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #15 from Dirk Brandewie <[email protected]> 2013-05-30 17:37:33 --- (In reply to comment #14)

Quoted 5 lines

I totally agree with Jean-Philippe: some information is lost with the 3.9 version and this information *is* useful to correctly set CPU frequencies. So either the related_cpus field meaning should be restored or a new field should be added.

Quoted 8 lines

If you doubt that such information is valuable, please consider all the runtime DVFS controllers such as beta adaptive [1], or CPU MISER [2]. Those systems, to be ported on current multicore processors, have to take a great care of the actually applied CPU frequency (not only that requested per core but the one actually in use). If it is impossible to determine what cores share the same frequency, it is impossible to perform such kind of DVFS on multicore CPUs in a portable way.

Exactly it is not possible to tell what frequency the core will run at unless the same pstate is requested for all cores. Even then it is not guaranteed in the presence of thermal throttling.

You do NOT have positive control over the frequency the core runs at. You requested your desired performance level and the processor ensures that you get it.

The papers below reference processors that did not have this functionality. the assumptions about frequency control no longer hold (at least for current Intel Processors).

Quoted 10 lines

If you think that related_cpus is misleading (and I'd totally agree with you on that), then maybe a new field is required.

[1] C.-h. Hsu and W.-c. Feng, “A power-aware run-time system for high-performance computing,” in Proceedings of the 2005 ACM/IEEE conference on Supercomputing.
[2] R. Ge, X. Feng, W. chun Feng, and K. Cameron, “CPU MISER: A performance-directed, run-time system for power-aware clusters,” in Parallel Processing, 2007. ICPP 2007.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#17 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

--- Comment #16 from Benoit Pradelle <[email protected]> 2013-05-30 20:00:30 --- (In reply to comment #15)

Quoted 19 lines

(In reply to comment #14)

Quoted 5 lines

I totally agree with Jean-Philippe: some information is lost with the 3.9 version and this information *is* useful to correctly set CPU frequencies. So either the related_cpus field meaning should be restored or a new field should be added.

Quoted 8 lines

If you doubt that such information is valuable, please consider all the runtime DVFS controllers such as beta adaptive [1], or CPU MISER [2]. Those systems, to be ported on current multicore processors, have to take a great care of the actually applied CPU frequency (not only that requested per core but the one actually in use). If it is impossible to determine what cores share the same frequency, it is impossible to perform such kind of DVFS on multicore CPUs in a portable way.

Exactly it is not possible to tell what frequency the core will run at unless the same pstate is requested for all cores. Even then it is not guaranteed in the presence of thermal throttling.

Quoted 8 lines

You do NOT have positive control over the frequency the core runs at. You requested your desired performance level and the processor ensures that you get it.

The papers below reference processors that did not have this functionality. the assumptions about frequency control no longer hold (at least for current Intel Processors).

OK I understand your point. However, even if in some cases (thermal threshold reached, TurboBoost, ...) the processors ignores/goes beyond the users requests, most of the time, when the user requests a frequency, it is effectively set by the processor. So, yes, the users still have some sort of control over CPU frequency in the general case (even if it went from precise frequency control to performance level requests) and it matters a lot when building and experimenting new DVFS controlers.

Going back to the topic, people are designing DVFS controlers and have to be aware of the cores using the same frequency in order to achieve efficient frequency control on multicore processors. A file was previously stating that relation so it should not be such a big deal to restore the same information in another file. Isn't it?

Quoted 11 lines
Quoted 11 lines

If you think that related_cpus is misleading (and I'd totally agree with you on that), then maybe a new field is required.

[1] C.-h. Hsu and W.-c. Feng, “A power-aware run-time system for high-performance computing,” in Proceedings of the 2005 ACM/IEEE conference on Supercomputing.
[2] R. Ge, X. Feng, W. chun Feng, and K. Cameron, “CPU MISER: A performance-directed, run-time system for power-aware clusters,” in Parallel Processing, 2007. ICPP 2007.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#18 lore raw
In reply to a message not in the archive

https://bugzilla.kernel.org/show_bug.cgi?id=58761

Lan Tianyu <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

                 CC|                            |[email protected]
         AssignedTo|[email protected]     |[email protected]
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.