ACPI / Processor: add sysfs support for low power idle
Subsystems (2)
RFC 1/1 ACPI / Processor: add sysfs support for low power idle
Add support to expose idle statistics maintained by platform to userspace via sysfs in addition to other data of interest from each LPI(Low Power Idle) state.
LPI described in section 8.4.4 of ACPI spec 6.1 provides different methods to obtain idle statistics maintained by the platform. These show a granular view of how each of the LPI state is being used at different level of hierarchy. sysfs data is exposed at each level in the hierarchy by creating a directory named 'lpi' at each level and the LPI state information is presented under it. Below is the representation of LPI information at one such level in the hierarchy
.../ACPI00XX: XX/lpi
|-> summary_stats |-> state0 | |-> desc | |-> time | |-> usage | |-> latency | |-> min_residency | |-> flags | |-> arch_flags | <<more states>>
ACPI00XX can be ACPI0007(processor) or ACPI0010(processor container)
stateX contains information related to a specific LPI state defined in the LPI ACPI tables.
summary_stats shows the stats(usage and time) from all the LPI states under a device. The summary_stats are provided to reduce the number' of files to be accessed by the userspace to capture a snapshot of the' idle statistics.
--- drivers/acpi/acpi_processor.c | 11 ++ drivers/acpi/processor_idle.c | 345 +++++++++++++++++++++++++++++++++++++++++- include/acpi/processor.h | 27 ++++ 3 files changed, 381 insertions(+), 2 deletions(-)
drivers/acpi/acpi_processor.c +11 -0
@@ -570,9 +570,19 @@ void __init acpi_early_processor_osc(void)static int acpi_processor_container_attach (struct acpi_device *dev,const struct acpi_device_id *id) { +if (dev->status.present && dev->status.functional && + dev->status.enabled && dev->status.show_in_ui) +acpi_lpi_sysfs_init (dev->handle, + (struct acpi_lpi_sysfs_data **)&dev->driver_data);return 1 ; } +static void acpi_processor_container_detach (struct acpi_device *dev) +{ +if (dev->driver_data) +acpi_lpi_sysfs_exit ((struct acpi_lpi_sysfs_data *)dev->driver_data); +} +static const struct acpi_device_id processor_container_ids[] = { {ACPI_PROCESSOR_CONTAINER_HID , }, { }
@@ -581,6 +591,7 @@ static int acpi_processor_container_attach(struct acpi_device *dev,static struct acpi_scan_handler processor_container_handler = { .ids = processor_container_ids, .attach = acpi_processor_container_attach, + .detach = acpi_processor_container_detach, };/* The number of the unique processor IDs */
drivers/acpi/processor_idle.c +343 -2
377 lines not on this page · raw message
include/acpi/processor.h +27 -0
@@ -79,6 +79,19 @@ struct acpi_lpi_state {u8 index;u8 entry_method;char desc[ACPI_CX_DESC_LEN ]; +struct acpi_generic_address res_cntr; +struct acpi_generic_address usage_cntr; +}; + +struct acpi_lpi_sysfs_state { +struct acpi_lpi_state *lpi_state; +struct kobject kobj; +}; + +struct acpi_lpi_sysfs_data { +u8 state_count; +struct kobject kobj; +struct acpi_lpi_sysfs_state *sysfs_states; };struct acpi_processor_power {
@@ -88,6 +101,7 @@ struct acpi_processor_power {struct acpi_lpi_state lpi_states[ACPI_PROCESSOR_MAX_POWER ]; };int timer_broadcast_on_state; +struct acpi_lpi_sysfs_data *lpi_sysfs_data; };/* Performance Management */
@@ -393,6 +407,8 @@ static inline void acpi_processor_throttling_init(void) {}int acpi_processor_power_exit (struct acpi_processor *pr);int acpi_processor_power_state_has_changed (struct acpi_processor *pr);int acpi_processor_hotplug (struct acpi_processor *pr); +int acpi_lpi_sysfs_init (acpi_handle h,struct acpi_lpi_sysfs_data **sysfs_data); +int acpi_lpi_sysfs_exit (struct acpi_lpi_sysfs_data *sysfs_data);#else static inline int acpi_processor_power_init (struct acpi_processor *pr) {
@@ -413,6 +429,17 @@ static inline int acpi_processor_hotplug(struct acpi_processor *pr) {return -ENODEV ; } + + +int acpi_lpi_sysfs_init (acpi_handle h,struct acpi_lpi_sysfs_data **sysfs_data) +{ +return -ENODEV ; +} + +int acpi_lpi_sysfs_exit (struct acpi_lpi_sysfs_data *sysfs_data) +{ +return -ENODEV ; +}#endif /* CONFIG_ACPI_PROCESSOR_IDLE */ /* in processor_thermal.c */
Signature · 3 lines
Qualcomm Datacenter Technologies on behalf of Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Quoted 30 lines · Prashanth Prakash
Add support to expose idle statistics maintained by platform to userspace via sysfs in addition to other data of interest from each LPI(Low Power Idle) state.
LPI described in section 8.4.4 of ACPI spec 6.1 provides different methods to obtain idle statistics maintained by the platform. These show a granular view of how each of the LPI state is being used at different level of hierarchy. sysfs data is exposed at each level in the hierarchy by creating a directory named 'lpi' at each level and the LPI state information is presented under it. Below is the representation of LPI information at one such level in the hierarchy
.../ACPI00XX: XX/lpi
|-> summary_stats |-> state0 | |-> desc | |-> time | |-> usage | |-> latency | |-> min_residency | |-> flags | |-> arch_flags | <<more states>>ACPI00XX can be ACPI0007(processor) or ACPI0010(processor container)
stateX contains information related to a specific LPI state defined in the LPI ACPI tables.
Quoted 6 lines
summary_statsshows the stats(usage and time) from all the LPI states under a device. Thesummary_statsare provided to reduce the number' of files to be accessed by the userspace to capture a snapshot of the' idle statistics.Signed-off-by Prashanth Prakash <[email protected]>
I'd like Sudeep to tell me what he thinks about this in the first place.
Quoted 11 lines
--- drivers/acpi/acpi_processor.c | 11 ++ drivers/acpi/processor_idle.c | 345 +++++++++++++++++++++++++++++++++++++++++- include/acpi/processor.h | 27 ++++ 3 files changed, 381 insertions(+), 2 deletions(-)drivers/acpi/acpi_processor.c +0
-0
drivers/acpi/acpi_processor.c void __init acpi_early_processor_osc(void) · line 576
Quoted 7 lines
drivers/acpi/acpi_processor.c +4
-0@@ -570,9 +570,19 @@ void __init acpi_early_processor_osc(void)static int acpi_processor_container_attach (struct acpi_device *dev,const struct acpi_device_id *id) { +if (dev->status.present && dev->status.functional && + dev->status.enabled && dev->status.show_in_ui) +acpi_lpi_sysfs_init (dev->handle, + (struct acpi_lpi_sysfs_data **)&dev->driver_data);
This isn't the right place to do it IMO.
It should be done at the processor driver initialization when it is know that LPI is going to be used at all.
drivers/acpi/acpi_processor.c void __init acpi_early_processor_osc(void) · line 583
Quoted 7 lines
drivers/acpi/acpi_processor.c +4
-0return 1 ; } +static void acpi_processor_container_detach (struct acpi_device *dev) +{ +if (dev->driver_data) +acpi_lpi_sysfs_exit ((struct acpi_lpi_sysfs_data *)dev->driver_data);
And analogously here.
drivers/acpi/acpi_processor.c void __init acpi_early_processor_osc(void) · line 588
Quoted 5 lines
drivers/acpi/acpi_processor.c +2
-0+} +static const struct acpi_device_id processor_container_ids[] = { {ACPI_PROCESSOR_CONTAINER_HID , }, { }
Thanks,
Rafael
Hi Rafael,
Quoted 59 lines · Rafael J. Wysocki
Quoted 30 lines · Prashanth Prakash
Add support to expose idle statistics maintained by platform to userspace via sysfs in addition to other data of interest from each LPI(Low Power Idle) state.
LPI described in section 8.4.4 of ACPI spec 6.1 provides different methods to obtain idle statistics maintained by the platform. These show a granular view of how each of the LPI state is being used at different level of hierarchy. sysfs data is exposed at each level in the hierarchy by creating a directory named 'lpi' at each level and the LPI state information is presented under it. Below is the representation of LPI information at one such level in the hierarchy
.../ACPI00XX: XX/lpi
|-> summary_stats |-> state0 | |-> desc | |-> time | |-> usage | |-> latency | |-> min_residency | |-> flags | |-> arch_flags | <<more states>>ACPI00XX can be ACPI0007(processor) or ACPI0010(processor container)
stateX contains information related to a specific LPI state defined in the LPI ACPI tables.
Quoted 6 lines
summary_statsshows the stats(usage and time) from all the LPI states under a device. Thesummary_statsare provided to reduce the number' of files to be accessed by the userspace to capture a snapshot of the' idle statistics.Signed-off-by Prashanth Prakash <[email protected]>I'd like Sudeep to tell me what he thinks about this in the first place.
Quoted 11 lines
--- drivers/acpi/acpi_processor.c | 11 ++ drivers/acpi/processor_idle.c | 345 +++++++++++++++++++++++++++++++++++++++++- include/acpi/processor.h | 27 ++++ 3 files changed, 381 insertions(+), 2 deletions(-)drivers/acpi/acpi_processor.c +0
-0drivers/acpi/acpi_processor.c void __init acpi_early_processor_osc(void) · line 576
Quoted 7 lines
drivers/acpi/acpi_processor.c +4
-0@@ -570,9 +570,19 @@ void __init acpi_early_processor_osc(void)static int acpi_processor_container_attach (struct acpi_device *dev,const struct acpi_device_id *id) { +if (dev->status.present && dev->status.functional && + dev->status.enabled && dev->status.show_in_ui) +acpi_lpi_sysfs_init (dev->handle, + (struct acpi_lpi_sysfs_data **)&dev->driver_data);This isn't the right place to do it IMO.
Quoted 2 lines
It should be done at the processor driver initialization when it is know that LPI is going to be used at all.
I will make the changes to initialize this in processor driver.
Thanks for the feedback!
--
Thanks,
Prashanth
Quoted 4 lines · Prashanth Prakash
Add support to expose idle statistics maintained by platform to userspace via sysfs in addition to other data of interest from each LPI(Low Power Idle) state.
While I understand this information is useful for some optimization and also for idle characterization with different workloads, I prefer to keep this in debugfs for:
1. We already have CPUIdle stats, this information may be more accurate which is good for above reasons but user-space applications shouldn't depend on it and end-up misusing it.
2. Also as more features get pushed into hardware, even these stats may not remain so much accurate as it is today and hence it would be better if user-space applications never use/depend on them.
Let me know if there are conflicting reasons ?
Quoted 10 lines
LPI described in section 8.4.4 of ACPI spec 6.1 provides different methods to obtain idle statistics maintained by the platform. These show a granular view of how each of the LPI state is being used at different level of hierarchy. sysfs data is exposed at each level in the hierarchy by creating a directory named 'lpi' at each level and the LPI state information is presented under it. Below is the representation of LPI information at one such level in the hierarchy
.../ACPI00XX: XX/lpi
|-> summary_stats
Not sure if it's any useful, see below.
Quoted 6 lines
|-> state0 | |-> desc | |-> time | |-> usage | |-> latency | |-> min_residency
The flags/arch_flags are meaning less if they are exposed as it. I would rather drop them.
Quoted 7 lines
| |-> flags | |-> arch_flags | <<more states>>ACPI00XX can be ACPI0007(processor) or ACPI0010(processor container)
Quoted 7 lines
stateX contains information related to a specific LPI state defined in the LPI ACPI tables.
summary_statsshows the stats(usage and time) from all the LPI states under a device. Thesummary_statsare provided to reduce the number' of files to be accessed by the userspace to capture a snapshot of the' idle statistics.
Really ? What's the need to reduce the no. of file accesses ?
Quoted 13 lines
Signed-off-by Prashanth Prakash <[email protected]>--- drivers/acpi/acpi_processor.c | 11 ++ drivers/acpi/processor_idle.c | 345 +++++++++++++++++++++++++++++++++++++++++- include/acpi/processor.h | 27 ++++ 3 files changed, 381 insertions(+), 2 deletions(-)drivers/acpi/acpi_processor.c +0
-0
drivers/acpi/acpi_processor.c void __init acpi_early_processor_osc(void) · line 576
Quoted 7 lines
drivers/acpi/acpi_processor.c +4
-0@@ -570,9 +570,19 @@ void __init acpi_early_processor_osc(void)static int acpi_processor_container_attach (struct acpi_device *dev,const struct acpi_device_id *id) { +if (dev->status.present && dev->status.functional && + dev->status.enabled && dev->status.show_in_ui) +acpi_lpi_sysfs_init (dev->handle, + (struct acpi_lpi_sysfs_data **)&dev->driver_data);
Why not register it only if LPIs are detected/initialized as active ?
[...]
drivers/acpi/acpi_processor.c void __init acpi_early_processor_osc(void) · line 584
Quoted 8 lines
drivers/acpi/acpi_processor.c +8
-0+ + +/* +* LPI sysfs support +* Exports two APIs that can be called as part of init and exit to setup the LPI +* sysfs entries either from processor or processor_container driver +*/ +
[...]
Lot of this sysfs handling looks similar to what we already have for cpuidle. I don't have a quick solution to avoid duplication but Greg really hate dealing with kobjects/ksets. I need to think if there's any better way to do this. Sorry for just raising issue without solution.
Quoted 7 lines
drivers/acpi/acpi_processor.c +7
-0+int acpi_lpi_sysfs_init (acpi_handle h, +struct acpi_lpi_sysfs_data **lpi_sysfs_data) +{ +struct acpi_device *d; +struct acpi_lpi_states_array info; +struct acpi_lpi_sysfs_state *sysfs_state =NULL ; +struct acpi_lpi_sysfs_data *data =NULL ;
drivers/acpi/acpi_processor.c void __init acpi_early_processor_osc(void)
Quoted 10 lines
drivers/acpi/acpi_processor.c +10
-0+int ret, i; + +if (!lpi_sysfs_data) +return -EINVAL ; + + ret =acpi_bus_get_device (h, &d); +if (ret) +return ret; + + ret =acpi_processor_evaluate_lpi (h, &info);
Why do we need to reevaluate _LPI here ?
Signature · 2 lines
Regards, Sudeep
Hi Sudeep,
Quoted 9 lines · Sudeep Holla
Quoted 4 lines · Prashanth Prakash
Add support to expose idle statistics maintained by platform to userspace via sysfs in addition to other data of interest from each LPI(Low Power Idle) state.
While I understand this information is useful for some optimization and also for idle characterization with different workloads, I prefer to keep this in debugfs for:
Quoted 8 lines
1. We already have CPUIdle stats, this information may be more accurate which is good for above reasons but user-space applications shouldn't depend on it and end-up misusing it.
2. Also as more features get pushed into hardware, even these stats may not remain so much accurate as it is today and hence it would be better if user-space applications never use/depend on them.Let me know if there are conflicting reasons ?
The information about idle state of shared resources(Cache, interconnect ...) which cannot be deduced from the cpuidle stats is quite useful. We can use this to analyze newer workloads and to explain their power consumption, especially as the amount of time some of these shared resources spends in different LPI states can be influenced by changes to workload, kernel or firmware. And for auto promote-able states this is the only way to capture idle stats.
Regarding 2, since these stats are clearly defined by ACPI spec and maintained by platform, I think it is reasonable to expect them to be accurate. If it is not accurate, it is likely that platform is breaking the spec.
Given above, I don't see much room for user-space applications to misuse this. Given these are defined as optional in spec, user-space application should use only if/when available and use it as complementary to cpuidle stats.
Sounds reasonable?
Quoted 12 lines
Quoted 10 lines
LPI described in section 8.4.4 of ACPI spec 6.1 provides different methods to obtain idle statistics maintained by the platform. These show a granular view of how each of the LPI state is being used at different level of hierarchy. sysfs data is exposed at each level in the hierarchy by creating a directory named 'lpi' at each level and the LPI state information is presented under it. Below is the representation of LPI information at one such level in the hierarchy
.../ACPI00XX: XX/lpi
|-> summary_statsNot sure if it's any useful, see below.
Quoted 8 lines
Quoted 6 lines
|-> state0 | |-> desc | |-> time | |-> usage | |-> latency | |-> min_residencyThe flags/arch_flags are meaning less if they are exposed as it. I would rather drop them.
Agree on arch_flags, will remove it.
We do decode the flags - Enabled/Disabled. I will make changes to populate sysfs only for enabled states and get rid of flags as well.
Quoted 8 lines
Quoted 8 lines
| |-> flags | |-> arch_flags | <<more states>>ACPI00XX can be ACPI0007(processor) or ACPI0010(processor container)
Quoted 8 lines
Quoted 7 lines
stateX contains information related to a specific LPI state defined in the LPI ACPI tables.
summary_statsshows the stats(usage and time) from all the LPI states under a device. Thesummary_statsare provided to reduce the number' of files to be accessed by the userspace to capture a snapshot of the' idle statistics.Really ? What's the need to reduce the no. of file accesses ?
When we have a large number of cores, with multiple idle state + few auto-promotable states. The amount of files we need to access to get a snapshot before/after a running a workload is quite high.
It gets worse if we want to keep the file handles open to sample it little more frequently, to get breakdown of idle state distribution during different phases of a workload.
Quoted 7 lines
Quoted 7 lines
Signed-off-by Prashanth Prakash <[email protected]>--- drivers/acpi/acpi_processor.c | 11 ++ drivers/acpi/processor_idle.c | 345 +++++++++++++++++++++++++++++++++++++++++- include/acpi/processor.h | 27 ++++ 3 files changed, 381 insertions(+), 2 deletions(-)
Quoted 13 lines
drivers/acpi/acpi_processor.c void __init acpi_early_processor_osc(void) · line 576
Quoted 12 lines
drivers/acpi/acpi_processor.c +4
-0@@ -570,9 +570,19 @@ void __init acpi_early_processor_osc(void)static int acpi_processor_container_attach (struct acpi_device *dev,const struct acpi_device_id *id) { +if (dev->status.present && dev->status.functional && + dev->status.enabled && dev->status.show_in_ui) +acpi_lpi_sysfs_init (dev->handle, + (struct acpi_lpi_sysfs_data **)&dev->driver_data);Why not register it only if LPIs are detected/initialized as active ?
Based on Rafael's feedback, I was planning to refactor the code for to initialize sysfs as part of LPI initialization, so that should take care of this.
Quoted 12 lines
[...]
Quoted 8 lines
drivers/acpi/acpi_processor.c +8
-0+ + +/* +* LPI sysfs support +* Exports two APIs that can be called as part of init and exit to setup the LPI +* sysfs entries either from processor or processor_container driver +*/ +[...]
Quoted 4 lines
Lot of this sysfs handling looks similar to what we already have for cpuidle. I don't have a quick solution to avoid duplication but Greg really hate dealing with kobjects/ksets. I need to think if there's any better way to do this. Sorry for just raising issue without solution.
Given the hierarchical nature of LPI and presence of auto promote-able states, it was hard to fit it to cpuidle model. I will take a look at it again, though I am not confident about finding a solution to avoid duplication.
Quoted 11 lines
Quoted 11 lines
drivers/acpi/acpi_processor.c +10
-0+int acpi_lpi_sysfs_init (acpi_handle h, +struct acpi_lpi_sysfs_data **lpi_sysfs_data) +{ +struct acpi_device *d; +struct acpi_lpi_states_array info; +struct acpi_lpi_sysfs_state *sysfs_state =NULL ; +struct acpi_lpi_sysfs_data *data =NULL ; +int ret, i; + +if (!lpi_sysfs_data)
Quoted 8 lines
drivers/acpi/acpi_processor.c
Quoted 7 lines
drivers/acpi/acpi_processor.c +7
-0+return -EINVAL ; + + ret =acpi_bus_get_device (h, &d); +if (ret) +return ret; + + ret =acpi_processor_evaluate_lpi (h, &info);Why do we need to reevaluate
_LPIhere ?
Since I am calling acpi_lpi_sysfs_init( ) from processor_driver and processor_container, calling reevaluate made the code much simpler.
Refactoring code to initialize LPI sysfs as part of LPI init in processor driver should likely remove the need to call reevaluate here.
Thanks a lot for your inputs!
--
Thanks,
Prashanth
Quoted 19 lines · Prashanth
Hi Sudeep,
Quoted 16 lines · Sudeep Holla
Quoted 4 lines · Prashanth Prakash
Add support to expose idle statistics maintained by platform to userspace via sysfs in addition to other data of interest from each LPI(Low Power Idle) state.
While I understand this information is useful for some optimization and also for idle characterization with different workloads, I prefer to keep this in debugfs for:
1. We already have CPUIdle stats, this information may be more accurate which is good for above reasons but user-space applications shouldn't depend on it and end-up misusing it.
2. Also as more features get pushed into hardware, even these stats may not remain so much accurate as it is today and hence it would be better if user-space applications never use/depend on them.
Quoted 8 lines
Quoted 1 line
Let me know if there are conflicting reasons ?
The information about idle state of shared
resources(Cache, interconnect ...)which cannot be deduced from the cpuidle stats is quite useful. We can use this to analyze newer workloads and to explain their power consumption, especially as the amount of time some of these shared resources spends in different LPI states can be influenced by changes to workload, kernel or firmware. And for auto promote-able states this is the only way to capture idle stats.
I agree that the stats are useful, no argument there. The question is more in terms of whether it can be debugfs which is just useful in analysis and characterization or sysfs which becomes user ABI.
Quoted 4 lines
Regarding 2, since these stats are clearly defined by ACPI spec and maintained by platform, I think it is reasonable to expect them to be accurate. If it is not accurate, it is likely that platform is breaking the spec.
I was considering firmware vs hardware here. If f/w tracks and updates these statistics, it may not be so accurate in the future if more controls that are today done in f/w will be done automatically done in h/w. If h/w updates these statistics, then yes they will be accurate.
I am still trying to convince myself if we need this as sysfs user ABI, so just thinking aloud.
Quoted 1 line
Given above, I don't see much room for user-space applications to misuse this.
You never know :)
Quoted 3 lines
Given these are defined as optional in spec, user-space application should use only if/when available and use it as complementary to cpuidle stats.
Fair enough.
[...]
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
| |-> flags | |-> arch_flags | <<more states>>ACPI00XX can be ACPI0007(processor) or ACPI0010(processor container)
stateX contains information related to a specific LPI state defined in the LPI ACPI tables.
Quoted 9 lines
Quoted 5 lines
Quoted 4 lines
summary_statsshows the stats(usage and time) from all the LPI states under a device. Thesummary_statsare provided to reduce the number' of files to be accessed by the userspace to capture a snapshot of the' idle statistics.Really ? What's the need to reduce the no. of file accesses ?
When we have a large number of cores, with multiple idle state + few auto-promotable states. The amount of files we need to access to get a snapshot before/after a running a workload is quite high.
OK. Since I don't have much knowledge on that, I can't really comment, but I wonder why is that not done for many stats that are per-cpu today.
Quoted 2 lines
It gets worse if we want to keep the file handles open to sample it little more frequently, to get breakdown of idle state distribution during different phases of a workload.
On the contrary agreeing with you on issue with large no. of file handles, why not we have single stat file that provides all the information you need and be done with it ? Why do we need all this hierarchy of sysfs if the summary_stats can provide all those information broken down.
[...]
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines
drivers/acpi/processor_idle.c +8
-0+ + +/* +* LPI sysfs support +* Exports two APIs that can be called as part of init and exit to setup the LPI +* sysfs entries either from processor or processor_container driver +*/ +[...]
Quoted 7 lines
Quoted 4 lines
Lot of this sysfs handling looks similar to what we already have for cpuidle. I don't have a quick solution to avoid duplication but Greg really hate dealing with kobjects/ksets. I need to think if there's any better way to do this. Sorry for just raising issue without solution.
Given the hierarchical nature of LPI and presence of auto promote-able states, it was hard to fit it to cpuidle model. I will take a look at it again, though I am not confident about finding a solution to avoid duplication.
Not a problem. I will also have a look, we can even do that later if required.
Signature · 2 lines
Regards, Sudeep
Quoted 33 lines · Sudeep Holla
Quoted 19 lines · Prashanth
Hi Sudeep,
Quoted 16 lines · Sudeep Holla
Quoted 4 lines · Prashanth Prakash
Add support to expose idle statistics maintained by platform to userspace via sysfs in addition to other data of interest from each LPI(Low Power Idle) state.
While I understand this information is useful for some optimization and also for idle characterization with different workloads, I prefer to keep this in debugfs for:
1. We already have CPUIdle stats, this information may be more accurate which is good for above reasons but user-space applications shouldn't depend on it and end-up misusing it.
2. Also as more features get pushed into hardware, even these stats may not remain so much accurate as it is today and hence it would be better if user-space applications never use/depend on them.Quoted 8 lines
Quoted 1 line
Let me know if there are conflicting reasons ?
The information about idle state of shared
resources(Cache, interconnect ...)which cannot be deduced from the cpuidle stats is quite useful. We can use this to analyze newer workloads and to explain their power consumption, especially as the amount of time some of these shared resources spends in different LPI states can be influenced by changes to workload, kernel or firmware. And for auto promote-able states this is the only way to capture idle stats.I agree that the stats are useful, no argument there. The question is more in terms of whether it can be debugfs which is just useful in analysis and characterization or sysfs which becomes user ABI.
Quoted 8 lines
Quoted 4 lines
Regarding 2, since these stats are clearly defined by ACPI spec and maintained by platform, I think it is reasonable to expect them to be accurate. If it is not accurate, it is likely that platform is breaking the spec.
I was considering firmware vs hardware here. If f/w tracks and updates these statistics, it may not be so accurate in the future if more controls that are today done in f/w will be done automatically done in h/w. If h/w updates these statistics, then yes they will be accurate.
Agree with f/w tracking vs h/w tracking, but I am not sure if we can (or should even try
to) handle those issues at LPI driver level.
Quoted 2 lines
I am still trying to convince myself if we need this as sysfs user ABI, so just thinking aloud.
Sure.
Quoted 2 lines
Quoted 1 line
Given above, I don't see much room for user-space applications to misuse this.
You never know :)
:-)
Quoted 22 lines
Quoted 4 lines
Given these are defined as optional in spec, user-space application should use only if/when available and use it as complementary to cpuidle stats.
Fair enough.
[...]
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
| |-> flags | |-> arch_flags | <<more states>>ACPI00XX can be ACPI0007(processor) or ACPI0010(processor container)
stateX contains information related to a specific LPI state defined in the LPI ACPI tables.
summary_statsshows the stats(usage and time) from all the LPI states under a device. Thesummary_statsare provided to reduce the number' of files to be accessed by the userspace to capture a snapshot of the' idle statistics.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Really ? What's the need to reduce the no. of file accesses ?
When we have a large number of cores, with multiple idle state + few auto-promotable states. The amount of files we need to access to get a snapshot before/after a running a workload is quite high.
OK. Since I don't have much knowledge on that, I can't really comment, but I wonder why is that not done for many stats that are per-cpu today.
Good point. The only example I can think of is cpufreq stats that does something like this, though I am not sure of exact motivation(s) behind it.
I think one could have made the similar case with cpuidle sysfs as well. Moreover summary_stats only mitigates some of the overhead with frequent reading of stats and doesn't really fix it in a scalable manner. So, I suppose the current reasoning to have summary_stats is quite weak to start with.
Quoted 7 lines
Quoted 2 lines
It gets worse if we want to keep the file handles open to sample it little more frequently, to get breakdown of idle state distribution during different phases of a workload.
On the contrary agreeing with you on issue with large no. of file handles, why not we have single stat file that provides all the information you need and be done with it ? Why do we need all this hierarchy of sysfs if the
summary_statscan provide all those information broken down.
With hierarchy, it is possible to get information about relationship between cpus and higher order shared resources. If we flatten the hierarchy we lose information about relationships and it gets a little hard to clearly represent the idle states of shared resources and which cpus share those resources.
--
Thanks,
Prashanth
Quoted 40 lines · Prashanth
Quoted 37 lines · Sudeep Holla
Quoted 19 lines · Prashanth
Hi Sudeep,
Quoted 16 lines · Sudeep Holla
Quoted 4 lines · Prashanth Prakash
Add support to expose idle statistics maintained by platform to userspace via sysfs in addition to other data of interest from each LPI(Low Power Idle) state.
While I understand this information is useful for some optimization and also for idle characterization with different workloads, I prefer to keep this in debugfs for:
1. We already have CPUIdle stats, this information may be more accurate which is good for above reasons but user-space applications shouldn't depend on it and end-up misusing it.
2. Also as more features get pushed into hardware, even these stats may not remain so much accurate as it is today and hence it would be better if user-space applications never use/depend on them.Quoted 8 lines
Quoted 1 line
Let me know if there are conflicting reasons ?
The information about idle state of shared
resources(Cache, interconnect ...)which cannot be deduced from the cpuidle stats is quite useful. We can use this to analyze newer workloads and to explain their power consumption, especially as the amount of time some of these shared resources spends in different LPI states can be influenced by changes to workload, kernel or firmware. And for auto promote-able states this is the only way to capture idle stats.I agree that the stats are useful, no argument there. The question is more in terms of whether it can be debugfs which is just useful in analysis and characterization or sysfs which becomes user ABI.
Quoted 4 lines
Regarding 2, since these stats are clearly defined by ACPI spec and maintained by platform, I think it is reasonable to expect them to be accurate. If it is not accurate, it is likely that platform is breaking the spec.
Quoted 7 lines
Quoted 4 lines
I was considering firmware vs hardware here. If f/w tracks and updates these statistics, it may not be so accurate in the future if more controls that are today done in f/w will be done automatically done in h/w. If h/w updates these statistics, then yes they will be accurate.
Agree with f/w tracking vs h/w tracking, but I am not sure if we can (or should even try
to) handle those issues at LPI driver level.
No I was not saying to control it. I just raised it to make sure we don't allow userspace to depend on these stats too much other than just characterization and analysis as these might be lost in future platforms.
Quoted 35 lines
Quoted 2 lines
I am still trying to convince myself if we need this as sysfs user ABI, so just thinking aloud.
Sure.
Quoted 2 lines
Quoted 1 line
Given above, I don't see much room for user-space applications to misuse this.
You never know :)
:-)
Quoted 22 lines
Quoted 4 lines
Given these are defined as optional in spec, user-space application should use only if/when available and use it as complementary to cpuidle stats.
Fair enough.
[...]
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
| |-> flags | |-> arch_flags | <<more states>>ACPI00XX can be ACPI0007(processor) or ACPI0010(processor container)
stateX contains information related to a specific LPI state defined in the LPI ACPI tables.
summary_statsshows the stats(usage and time) from all the LPI states under a device. Thesummary_statsare provided to reduce the number' of files to be accessed by the userspace to capture a snapshot of the' idle statistics.Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Really ? What's the need to reduce the no. of file accesses ?
When we have a large number of cores, with multiple idle state + few auto-promotable states. The amount of files we need to access to get a snapshot before/after a running a workload is quite high.
OK. Since I don't have much knowledge on that, I can't really comment, but I wonder why is that not done for many stats that are per-cpu today.
Quoted 7 lines
Good point. The only example I can think of is cpufreq stats that does something like this, though I am not sure of exact motivation(s) behind it.
I think one could have made the similar case with cpuidle sysfs as well. Moreover
summary_statsonly mitigates some of the overhead with frequent reading of stats and doesn't really fix it in a scalable manner. So, I suppose the current reasoning to havesummary_statsis quite weak to start with.
After you brought up this point, I was thinking to use this over hierarchical representation. If we put this summary_stats in debugfs, then no need to worry about user ABI.
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
It gets worse if we want to keep the file handles open to sample it little more frequently, to get breakdown of idle state distribution during different phases of a workload.
On the contrary agreeing with you on issue with large no. of file handles, why not we have single stat file that provides all the information you need and be done with it ? Why do we need all this hierarchy of sysfs if the
summary_statscan provide all those information broken down.
Quoted 5 lines
With hierarchy, it is possible to get information about relationship between cpus and higher order shared resources. If we flatten the hierarchy we lose information about relationships and it gets a little hard to clearly represent the idle states of shared resources and which cpus share those resources.
No, what I meant is to have complete information broken down to the lowest possible level, but just one file access to get that information.
IIUC, you had some format already for summary_stat file, just extend to get all the information you would get reading individual sysfs files organized hierarchically. I don't think that should matter much as you would have all these information, just the representation differs. I am assuming all this data is not used/interpreted on the fly, but mostly used offline for analysis. Otherwise it's already misuse and we don't want to expose such user ABI IMO.
Just to summarize, though I agree this LPI stats are more accurate and more representative summary, I think it may fade away as we move towards hardware controlled lower power states. Since we already have cpuidle stats, I prefer to keep this LPI stats interface to userspace as simple and minimal as possible and yet helpful to get all the information.
Signature · 2 lines
Regards, Sudeep
Hi Sudeep,
Quoted 16 lines · Sudeep Holla
Quoted 5 lines
With hierarchy, it is possible to get information about relationship between cpus and higher order shared resources. If we flatten the hierarchy we lose information about relationships and it gets a little hard to clearly represent the idle states of shared resources and which cpus share those resources.
No, what I meant is to have complete information broken down to the lowest possible level, but just one file access to get that information.
IIUC, you had some format already for
summary_statfile, just extend to get all the information you would get reading individual sysfs files organized hierarchically. I don't think that should matter much as you would have all these information, just the representation differs. I am assuming all this data is not used/interpreted on the fly, but mostly used offline for analysis. Otherwise it's already misuse and we don't want to expose such user ABI IMO.
Quoted 5 lines
Just to summarize, though I agree this LPI stats are more accurate and more representative summary, I think it may fade away as we move towards hardware controlled lower power states. Since we already have cpuidle stats, I prefer to keep this LPI stats interface to userspace as simple and minimal as possible and yet helpful to get all the information.
Sounds good. I will remove summary_stats and make other changes based on feedback and post next rev.
Thanks for the review and feedback!
--
Thanks,
Prashanth
Quoted 19 lines · Prashanth
Hi Sudeep,
Quoted 16 lines · Sudeep Holla
Quoted 5 lines
With hierarchy, it is possible to get information about relationship between cpus and higher order shared resources. If we flatten the hierarchy we lose information about relationships and it gets a little hard to clearly represent the idle states of shared resources and which cpus share those resources.
No, what I meant is to have complete information broken down to the lowest possible level, but just one file access to get that information.
IIUC, you had some format already for
summary_statfile, just extend to get all the information you would get reading individual sysfs files organized hierarchically. I don't think that should matter much as you would have all these information, just the representation differs. I am assuming all this data is not used/interpreted on the fly, but mostly used offline for analysis. Otherwise it's already misuse and we don't want to expose such user ABI IMO.
Quoted 8 lines
Quoted 5 lines
Just to summarize, though I agree this LPI stats are more accurate and more representative summary, I think it may fade away as we move towards hardware controlled lower power states. Since we already have cpuidle stats, I prefer to keep this LPI stats interface to userspace as simple and minimal as possible and yet helpful to get all the information.
Sounds good. I will remove
summary_statsand make other changes based on feedback and post next rev.
I am sorry if I was not clear earlier, I was infact asking you what if we just have single summary_stat file with all statistics broken down to minuet details you want instead of creating the hierarchy of individual files which you say has overhead to read.
Do you process the stats on the fly when workload is running, I assume not.
Signature · 2 lines
Regards, Sudeep
Quoted 26 lines · Sudeep Holla
Quoted 24 lines · Prashanth
Hi Sudeep,
Quoted 21 lines · Sudeep Holla
Quoted 5 lines
With hierarchy, it is possible to get information about relationship between cpus and higher order shared resources. If we flatten the hierarchy we lose information about relationships and it gets a little hard to clearly represent the idle states of shared resources and which cpus share those resources.
No, what I meant is to have complete information broken down to the lowest possible level, but just one file access to get that information.
IIUC, you had some format already for
summary_statfile, just extend to get all the information you would get reading individual sysfs files organized hierarchically. I don't think that should matter much as you would have all these information, just the representation differs. I am assuming all this data is not used/interpreted on the fly, but mostly used offline for analysis. Otherwise it's already misuse and we don't want to expose such user ABI IMO.Just to summarize, though I agree this LPI stats are more accurate and more representative summary, I think it may fade away as we move towards hardware controlled lower power states. Since we already have cpuidle stats, I prefer to keep this LPI stats interface to userspace as simple and minimal as possible and yet helpful to get all the information.
Quoted 8 lines
Quoted 2 lines
Sounds good. I will remove
summary_statsand make other changes based on feedback and post next rev.I am sorry if I was not clear earlier, I was infact asking you what if we just have single
summary_statfile with all statistics broken down to minuet details you want instead of creating the hierarchy of individual files which you say has overhead to read.Do you process the stats on the fly when workload is running, I assume not.
I don't see any use-cases for on the fly analysis. The most we might want to do it capture this data periodically and then post-process it to see how different phases of a workload did. The overhead of reading multiple files comes down what could be the duration between reads to this sysfs without drastically impacting the workload - so it is probably not such a big issue.
While it is possible to simplify by having all the data aggregated into a single file at each level in the hierarchy, we end up trading some inefficiencies:
- We are reducing the number of file access but increasing the amount of data read (which user-space may or may not be interested in), including re-reading the constant fields(desc, latency etc..) every time.
- Adding all info on the same file increases the probability we might break userspace parsing of this single file when we want add a new piece of information. While this was possible earlier, the probability was little lower as it was restricted to only time and usage.
- Another reason to avoid collapsing all the fields to single file is to keep it consistent with cpudile interface
I was thinking about some of the above issues and when you mentioned simple and minimal, I interpreted it as lets keep the interface simple by removing summary_stats and few other fields we discussed. Sorry, my bad.
--
Thanks,
Prashanth