Energy-aware Scheduling Workshop
Hi,
Last year's Energy-Aware Scheduling workshop [1,2] was a good opportunity for interested parties to discuss some of the open issues in this area face to face. While work is still ongoing on many of the topics that were discussed, it might be worth having workshop again this year to follow up, revise the plans if necessary, and discuss topics that were not covered last year.
Before submitting a workshop proposal to the Ksummit PC I would like to probe the interest. IMO, it is important that we have scheduler maintainers present.
Workshop topic proposals:
Test cases
Use-cases for high-end phones (which some of us care about) consist of rather complex software stacks which are not suitable for quick patch testing [3]. While we can't avoid testing using the full software stack in the end, it would be useful to have configurable micro-benchmarks for initial testing and to reproduce specific scheduling patterns from the full use-case for debugging purposes.
Energy Evaluation
A hot topic last year. We need a way to evaluate energy-awareness patches. Work has started on an idle state analysis tool [4], but we are not there yet.
Platform Performance/Energy data
Currently the kernel has quite limited knowledge about energy costs of the platform where it is running. Without this information it is rather hard to make energy-efficient scheduling decisions. It seems that various energy-saving techniques don't work equally well on all platforms and might even depend on the use-case. Should we give the kernel enough information to construct a simple energy-model to guide decisions?
CPU utilization and cpu_power
The entity load tracking has given us a much better indication of individual task loadi. However, priority scaling makes it less suitable for low load scenarios [5] where we care more about actual cpu utilization per task when trying to figure out an energy-efficient load balance. Do we need entity utilization tracking as well? Related to this topic is the representation of cpu compute capacity. The current representation, cpu_power, can't deal with heterogeneous systems correctly. Can we come up with a solution that can handle SMT, SMP, and heterogeneous systems?
All comments and topic proposals are welcome.
Thanks,
Morten
[1] http://lwn.net/Articles/571414/
[2] http://etherpad.osuosl.org/energy-aware-scheduling-ks-2013
[3] https://lkml.org/lkml/2014/1/7/355
[4] https://lkml.org/lkml/2014/3/24/363
[5] https://lkml.org/lkml/2014/1/7/503
Quoted 1 line · Morten Rasmussen
Hi,
Hi,
Quoted 47 lines
Last year's Energy-Aware Scheduling workshop [1,2] was a good opportunity for interested parties to discuss some of the open issues in this area face to face. While work is still ongoing on many of the topics that were discussed, it might be worth having workshop again this year to follow up, revise the plans if necessary, and discuss topics that were not covered last year.
Before submitting a workshop proposal to the Ksummit PC I would like to probe the interest. IMO, it is important that we have scheduler maintainers present.
Workshop topic proposals:
Test cases
Use-cases for high-end phones (which some of us care about) consist of rather complex software stacks which are not suitable for quick patch testing [3]. While we can't avoid testing using the full software stack in the end, it would be useful to have configurable micro-benchmarks for initial testing and to reproduce specific scheduling patterns from the full use-case for debugging purposes.Energy Evaluation
A hot topic last year. We need a way to evaluate energy-awareness patches. Work has started on an idle state analysis tool [4], but we are not there yet.Platform Performance/Energy data
Currently the kernel has quite limited knowledge about energy costs of the platform where it is running. Without this information it is rather hard to make energy-efficient scheduling decisions. It seems that various energy-saving techniques don't work equally well on all platforms and might even depend on the use-case. Should we give the kernel enough information to construct a simple energy-model to guide decisions?CPU utilization and
cpu_power
The entity load tracking has given us a much better indication of individual task loadi. However, priority scaling makes it less suitable for low load scenarios [5] where we care more about actual cpu utilization per task when trying to figure out an energy-efficient load balance. Do we need entity utilization tracking as well? Related to this topic is the representation of cpu compute capacity. The current representation,cpu_power, can't deal with heterogeneous systems correctly. Can we come up with a solution that can handle SMT, SMP, and heterogeneous systems?
Quoted 1 line
All comments and topic proposals are welcome.
I would be interested in participating in that discussion (which also is related to the energy conservation bias interfaces KS topic proposed by me).
Thanks!
Quoted 5 lines
[1] http://lwn.net/Articles/571414/
[2] http://etherpad.osuosl.org/energy-aware-scheduling-ks-2013
[3] https://lkml.org/lkml/2014/1/7/355
[4] https://lkml.org/lkml/2014/3/24/363
[5] https://lkml.org/lkml/2014/1/7/503
Signature · 2 lines
I speak only for myself. Rafael J. Wysocki, Intel Open Source Technology Center.
Hi Morten,
Thank you for initiating this.
Quoted 23 lines · Morten Rasmussen
Hi,
Last year's Energy-Aware Scheduling workshop [1,2] was a good opportunity for interested parties to discuss some of the open issues in this area face to face. While work is still ongoing on many of the topics that were discussed, it might be worth having workshop again this year to follow up, revise the plans if necessary, and discuss topics that were not covered last year.
Before submitting a workshop proposal to the Ksummit PC I would like to probe the interest. IMO, it is important that we have scheduler maintainers present.
Workshop topic proposals:
Test cases
Use-cases for high-end phones (which some of us care about) consist of rather complex software stacks which are not suitable for quick patch testing [3]. While we can't avoid testing using the full software stack in the end, it would be useful to have configurable micro-benchmarks for initial testing and to reproduce specific scheduling patterns from the full use-case for debugging purposes.
Quoted 5 lines
Energy Evaluation
A hot topic last year. We need a way to evaluate energy-awareness patches. Work has started on an idle state analysis tool [4], but we are not there yet.
An interesting sub topic pertaining to evaluation would be deciding the target residency of idle states. How do we know if the target residency set for the different idle states is set optimally? The cpuidle governor bases its decisions of the idle state to enter into based on numbers such as these. If it is too stringent we would be hampering power savings.
We could run different benchmarks with different degree of utilization. And observe how much of the idle time is spent in different idle states. If it is too less in a particular idle state we can up the target residency of that idle state to begin with. This is static setting and we hard code the target residency like now. Will a dynamic setting of metrics like these help us in any way?
This was just an example to show that there are interesting problems around "tuning" of power management metrics. It would be very helpful if the maintainers relevant to this workshop guide us on this.
Quoted 21 lines
Platform Performance/Energy data
Currently the kernel has quite limited knowledge about energy costs of the platform where it is running. Without this information it is rather hard to make energy-efficient scheduling decisions. It seems that various energy-saving techniques don't work equally well on all platforms and might even depend on the use-case. Should we give the kernel enough information to construct a simple energy-model to guide decisions?CPU utilization and
cpu_power
The entity load tracking has given us a much better indication of individual task loadi. However, priority scaling makes it less suitable for low load scenarios [5] where we care more about actual cpu utilization per task when trying to figure out an energy-efficient load balance. Do we need entity utilization tracking as well? Related to this topic is the representation of cpu compute capacity. The current representation,cpu_power, can't deal with heterogeneous systems correctly. Can we come up with a solution that can handle SMT, SMP, and heterogeneous systems?
Quoted 1 line
All comments and topic proposals are welcome.
I would be very interested in attending this workshop. We are currently evaluating cpuidle and cpu-frequency subsystems on PowerPC. I look forward to share and discuss the findings in this workshop.
Thanks!
Regards
Preeti U Murthy
Quoted 10 lines
Thanks,
Morten[1] http://lwn.net/Articles/571414/
[2] http://etherpad.osuosl.org/energy-aware-scheduling-ks-2013
[3] https://lkml.org/lkml/2014/1/7/355
[4] https://lkml.org/lkml/2014/3/24/363
[5] https://lkml.org/lkml/2014/1/7/503
Quoted 40 lines · Preeti U Murthy
Hi Morten,
Thank you for initiating this.
Quoted 23 lines · Morten Rasmussen
Hi,
Last year's Energy-Aware Scheduling workshop [1,2] was a good opportunity for interested parties to discuss some of the open issues in this area face to face. While work is still ongoing on many of the topics that were discussed, it might be worth having workshop again this year to follow up, revise the plans if necessary, and discuss topics that were not covered last year.
Before submitting a workshop proposal to the Ksummit PC I would like to probe the interest. IMO, it is important that we have scheduler maintainers present.
Workshop topic proposals:
Test cases
Use-cases for high-end phones (which some of us care about) consist of rather complex software stacks which are not suitable for quick patch testing [3]. While we can't avoid testing using the full software stack in the end, it would be useful to have configurable micro-benchmarks for initial testing and to reproduce specific scheduling patterns from the full use-case for debugging purposes.Quoted 5 lines
Energy Evaluation
A hot topic last year. We need a way to evaluate energy-awareness patches. Work has started on an idle state analysis tool [4], but we are not there yet.An interesting sub topic pertaining to evaluation would be deciding the target residency of idle states. How do we know if the target residency set for the different idle states is set optimally? The cpuidle governor bases its decisions of the idle state to enter into based on numbers such as these. If it is too stringent we would be hampering power savings.
Quoted 6 lines
We could run different benchmarks with different degree of utilization. And observe how much of the idle time is spent in different idle states. If it is too less in a particular idle state we can up the target residency of that idle state to begin with. This is static setting and we hard code the target residency like now. Will a dynamic setting of metrics like these help us in any way?
Preeti,
We've created a tool called idlestat[1] that traces the cpuidle and cpufreq code paths and then creates a report about how often each state was reached, average residency, etc.
We're currently experimenting with a patch to output how often we mispredict the idle state and are seeing some good corelation with actual power measurements. Dynamic residency might be a solution, but first we need to understand how bad the problem is. That is what this tool is designed to solve. It also helps provide a qualitative feel for kernel behaviour without requiring HW hacking to measure various power rails.
Daniel and I'd be interested in talking about idlestat and share our findings about using it to study change in kernel behaviour due to certain patchsets.
Regards,
Amit
[1] https://git.linaro.org/power/idlestat.git
Hi Preeti,
Quoted 40 lines · Preeti U Murthy
Hi Morten,
Thank you for initiating this.
Quoted 23 lines · Morten Rasmussen
Hi,
Last year's Energy-Aware Scheduling workshop [1,2] was a good opportunity for interested parties to discuss some of the open issues in this area face to face. While work is still ongoing on many of the topics that were discussed, it might be worth having workshop again this year to follow up, revise the plans if necessary, and discuss topics that were not covered last year.
Before submitting a workshop proposal to the Ksummit PC I would like to probe the interest. IMO, it is important that we have scheduler maintainers present.
Workshop topic proposals:
Test cases
Use-cases for high-end phones (which some of us care about) consist of rather complex software stacks which are not suitable for quick patch testing [3]. While we can't avoid testing using the full software stack in the end, it would be useful to have configurable micro-benchmarks for initial testing and to reproduce specific scheduling patterns from the full use-case for debugging purposes.Quoted 5 lines
Energy Evaluation
A hot topic last year. We need a way to evaluate energy-awareness patches. Work has started on an idle state analysis tool [4], but we are not there yet.An interesting sub topic pertaining to evaluation would be deciding the target residency of idle states. How do we know if the target residency set for the different idle states is set optimally? The cpuidle governor bases its decisions of the idle state to enter into based on numbers such as these. If it is too stringent we would be hampering power savings.
Quoted 6 lines
We could run different benchmarks with different degree of utilization. And observe how much of the idle time is spent in different idle states. If it is too less in a particular idle state we can up the target residency of that idle state to begin with. This is static setting and we hard code the target residency like now. Will a dynamic setting of metrics like these help us in any way?
It might, as long as you are able to monitor energy in the kernel. target_residency is expressed in time, but must be evaluated in energy terms, it is a break-even point. It strictly depends on the dynamic state of the CPU when it goes idle, hence not really easy to factor in.
On ARM, it depends on a slew of factors, inclusive of the number of dirty caches lines (ie memory writebacks), running CPU frequency, memory frequency and the list goes on and on and on.
I am not even taking into account CPU idle states (C-states) whose power domains span devices such as GPUs.
It is certainly an interesting topic, I gave a presentation on this at Linaro Connect in 2013, I would be happy to share the results and debate it if I am able to attend.
Quoted 3 lines
This was just an example to show that there are interesting problems around "tuning" of power management metrics. It would be very helpful if the maintainers relevant to this workshop guide us on this.
I would be certainly happy to explain to you how idle states work on ARM systems and the factors contributing to target_residency and other idle states parameters.
Thanks,
Lorenzo
Hi Amit, Lorenzo,
Quoted 66 lines · Lorenzo Pieralisi
Hi Preeti,
Quoted 40 lines · Preeti U Murthy
Hi Morten,
Thank you for initiating this.
Quoted 23 lines · Morten Rasmussen
Hi,
Last year's Energy-Aware Scheduling workshop [1,2] was a good opportunity for interested parties to discuss some of the open issues in this area face to face. While work is still ongoing on many of the topics that were discussed, it might be worth having workshop again this year to follow up, revise the plans if necessary, and discuss topics that were not covered last year.
Before submitting a workshop proposal to the Ksummit PC I would like to probe the interest. IMO, it is important that we have scheduler maintainers present.
Workshop topic proposals:
Test cases
Use-cases for high-end phones (which some of us care about) consist of rather complex software stacks which are not suitable for quick patch testing [3]. While we can't avoid testing using the full software stack in the end, it would be useful to have configurable micro-benchmarks for initial testing and to reproduce specific scheduling patterns from the full use-case for debugging purposes.Quoted 5 lines
Energy Evaluation
A hot topic last year. We need a way to evaluate energy-awareness patches. Work has started on an idle state analysis tool [4], but we are not there yet.An interesting sub topic pertaining to evaluation would be deciding the target residency of idle states. How do we know if the target residency set for the different idle states is set optimally? The cpuidle governor bases its decisions of the idle state to enter into based on numbers such as these. If it is too stringent we would be hampering power savings.
Quoted 6 lines
We could run different benchmarks with different degree of utilization. And observe how much of the idle time is spent in different idle states. If it is too less in a particular idle state we can up the target residency of that idle state to begin with. This is static setting and we hard code the target residency like now. Will a dynamic setting of metrics like these help us in any way?
It might, as long as you are able to monitor energy in the kernel.
target_residencyis expressed in time, but must be evaluated in energy terms, it is a break-even point. It strictly depends on the dynamic state of the CPU when it goes idle, hence not really easy to factor in.On ARM, it depends on a slew of factors, inclusive of the number of dirty caches lines (ie memory writebacks), running CPU frequency, memory frequency and the list goes on and on and on.
I am not even taking into account CPU idle states (C-states) whose power domains span devices such as GPUs.
It is certainly an interesting topic, I gave a presentation on this at Linaro Connect in 2013, I would be happy to share the results and debate it if I am able to attend.
Quoted 7 lines
Quoted 3 lines
This was just an example to show that there are interesting problems around "tuning" of power management metrics. It would be very helpful if the maintainers relevant to this workshop guide us on this.
I would be certainly happy to explain to you how idle states work on ARM systems and the factors contributing to
target_residencyand other idle states parameters.
Thanks for sharing your ideas about this.
Lorenzo,
Sure I will be glad if we are able to meet to discuss this.
Amit,
I will take a look at idle stat and verify if I can use it for my purpose.
Regards
Preeti U Murthy
Quoted 4 lines
Thanks,
Lorenzo