powercap/dtpm: Add the DTPM framework
Subsystems (5)
v2 cover powercap/dtpm: Add the DTPM framework
The density of components greatly increased the last decade bringing a numerous number of heating sources which are monitored by more than 20 sensors on recent SoC. The skin temperature, which is the case temperature of the device, must stay below approximately 45°C in order to comply with the legal requirements.
The skin temperature is managed as a whole by an user space daemon, which is catching the current application profile, to allocate a power budget to the different components where the resulting heating effect will comply with the skin temperature constraint.
This technique is called the Dynamic Thermal Power Management.
The Linux kernel does not provide any unified interface to act on the power of the different devices. Currently, the thermal framework is changed to export artificially the performance states of different devices via the cooling device software component with opaque values. This change is done regardless of the in-kernel logic to mitigate the temperature. The user space daemon uses all the available knobs to act on the power limit and those differ from one platform to another.
This series provides a Dynamic Thermal Power Management framework to provide an unified way to act on the power of the devices.
Changelog:
V2:
- Fixed indentation
- Fixed typos in comments
- Fixed missing kfree for dtpm_cpu
- Capitalize letters in the Kconfig description
- Reduced name description
- Stringified section name
- Added more debug traces in the code
- Removed duplicate initialization in the dtpm cpu
Daniel Lezcano (4):
units: Add Watt units
Documentation/powercap/dtpm: Add documentation for dtpm powercap/drivers/dtpm: Add API for dynamic thermal power management powercap/drivers/dtpm: Add CPU energy model based support
Documentation/power/powercap/dtpm.rst | 222 +++++++++++++ drivers/powercap/Kconfig | 13 + drivers/powercap/Makefile | 2 + drivers/powercap/dtpm.c | 436 ++++++++++++++++++++++++++ drivers/powercap/dtpm_cpu.c | 282 +++++++++++++++++ include/asm-generic/vmlinux.lds.h | 11 + include/linux/cpuhotplug.h | 1 + include/linux/dtpm.h | 75 +++++ include/linux/units.h | 4 + 9 files changed, 1046 insertions(+) create mode 100644 Documentation/power/powercap/dtpm.rst create mode 100644 drivers/powercap/dtpm.c create mode 100644 drivers/powercap/dtpm_cpu.c create mode 100644 include/linux/dtpm.h
Signature · 1 line
2.17.1
PATCH v2 1/4 units: Add Watt units
As there are the temperature units, let's add the Watt macros definition.
--- include/linux/units.h | 4 ++++ 1 file changed, 4 insertions(+)
include/linux/units.h +4 -0
@@ -4,6 +4,10 @@#include <linux/kernel.h> +#define MILLIWATT_PER_WATT 1000L +#define MICROWATT_PER_MILLIWATT 1000L +#define MICROWATT_PER_WATT 1000000L +#define ABSOLUTE_ZERO_MILLICELSIUS -273150static inline long milli_kelvin_to_millicelsius (long t)
Signature · 1 line
2.17.1
PATCH v2 2/4 Documentation/powercap/dtpm: Add documentation for dtpm
In reply to Daniel LezcanoThe dynamic thermal and power management is a technique to dynamically adjust the power consumption of different devices in order to ensure a global thermal constraint.
An userspace daemon is usually monitoring the temperature and the power to take immediate action on the device.
The DTPM framework provides an unified API to userspace to act on the power.
Document this framework.
--- Documentation/power/powercap/dtpm.rst | 222 ++++++++++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 Documentation/power/powercap/dtpm.rst
Documentation/power/powercap/dtpm.rst new file +222 -0
@@ -0,0 +1,222 @@ +========================================== +Dynamic Thermal Power Management framework +========================================== + +On the embedded world, the complexity of the SoC leads to an +increasing number of hotspots which need to be monitored and mitigated +as a whole in order to prevent the temperature to go above the +normative and legally stated 'skin temperature'. + +Another aspect is to sustain the performance for a given power budget, +for example virtual reality where the user can feel dizziness if the +performance is capped while a big CPU is processing something else. Or +reduce the battery charging because the dissipated power is too high +compared with the power consumed by other devices. + +The userspace is the most adequate place to dynamically act on the +different devices by limiting their power given an application +profile: it has the knowledge of the platform. + +The Dynamic Thermal Power Management (DTPM) is a technique acting on +the device power by limiting and/or balancing a power budget among +different devices. + +The DTPM framework provides an unified interface to act on the +device power. + +=========== +1. Overview +=========== + +The DTPM framework relies on the powercap framework to create the +powercap entries in the sysfs directory and implement the backend +driver to do the connection with the power manageable device. + +The DTPM is a tree representation describing the power constraints +shared between devices, not their physical positions. + +The nodes of the tree are a virtual description aggregating the power +characteristics of the children nodes and their power limitations. + +The leaves of the tree are the real power manageable devices. + +For instance: + + SoC + | + `-- pkg + | + |-- pd0 (cpu0-3) + | + `-- pd1 (cpu4-5) + +* The pkg power will be the sum of pd0 and pd1 power numbers. + + SoC (400mW - 3100mW) + | + `-- pkg (400mW - 3100mW) + | + |-- pd0 (100mW - 700mW) + | + `-- pd1 (300mW - 2400mW) + +* When the nodes are inserted in the tree, their power characteristics + are propagated to the parents. + + SoC (600mW - 5900mW) + | + |-- pkg (400mW - 3100mW) + | | + | |-- pd0 (100mW - 700mW) + | | + | `-- pd1 (300mW - 2400mW) + | + `-- pd2 (200mW - 2800mW) + +* Each node have a weight on a 2^10 basis reflecting the percentage of + power consumption along the siblings. + + SoC (w=1024) + | + |-- pkg (w=538) + | | + | |-- pd0 (w=231) + | | + | `-- pd1 (w=794) + | + `-- pd2 (w=486) + + Note the sum of weights at the same level are equal to 1024. + +* When a power limitation is applied to a node, then it is distributed + along the children given their weights. For example, if we set a + power limitation of 3200mW at the 'SoC' root node, the resulting + tree will be. + + SoC (w=1024) <--- power_limit = 3200mW + | + |-- pkg (w=538) --> power_limit = 1681mW + | | + | |-- pd0 (w=231) --> power_limit = 378mW + | | + | `-- pd1 (w=794) --> power_limit = 1303mW + | + `-- pd2 (w=486) --> power_limit = 1519mW + +==================== +1.1 Flat description +==================== + +A root node is created and it is the parent of all the nodes. This +description is the simplest one and it is supposed to give to +userspace a flat representation of all the devices supporting the +power limitation without any power limitation distribution. + +============================ +1.2 Hierarchical description +============================ + +The different devices supporting the power limitation are represented +hierarchically. There is one root node, all intermediate nodes are +grouping the child nodes which can be intermediate nodes also or real +devices. + +The intermediate nodes aggregate the power information and allows to +set the power limit given the weight of the nodes. + +================ +2. Userspace API +================ + +As stated in the overview, the DTPM framework is built on top of the +powercap framework. Thus the sysfs interface is the same, please refer +to the powercap documentation for further details. + + * power_uw: Instantaneous power consumption. If the node is an + intermediate node, then the power consumption will be the sum of all + children power consumption. + + * max_power_range_uw: The power range resulting of the maximum power + minus the minimum power. + + * name: The name of the node. This is implementation dependant. Even + if it is not recommended for the userspace, several nodes can have + the same name. + + * constraint_X_name: The name of the constraint. + + * constraint_X_max_power_uw: The maximum power limit to be applicable + to the node. + + * constraint_X_power_limit_uw: The power limit to be applied to the + node. If the value contained in constraint_X_max_power_uw is set, + the constraint will be removed. + + * constraint_X_time_window_us: The meaning of this file will depend + on the constraint number. + +=============== +2.1 Constraints +=============== + + * Constraint 0: The power limitation is immediately applied, without + limitation in time. + +============= +3. Kernel API +============= + +============ +3.1 Overview +============ + +The DTPM framework has no power limiting backend support. It is +generic and provides a set of API to let the different drivers to +implement the backend part for the power limitation and create a the +power constraints tree. + +It is up to the platform to provide the initialization function to +allocate and link the different nodes of the tree. + +A special macro has the role of declaring a node and the corresponding +initialization function via a description structure. This one contains +an optional parent field allowing to hook different devices to an +already existing tree at boot time. + +struct dtpm_descr my_descr = { + .name = "my_name", + .init = my_init_func, +}; + +DTPM_DECLARE(my_descr); + +The nodes of the DTPM tree are described with dtpm structure. The +steps to add a new power limitable device is done in three steps: + + * Allocate the dtpm node + * Set the power number of the dtpm node + * Register the dtpm node + +The registration of the dtpm node is done with the powercap +ops. Basically, it must implements the callbacks to get and set the +power and the limit. + +Alternatively, if the node to be inserted is an intermediate one, then +a simple function to insert it as a future parent is available. + +If a device has its power characteristics changing, then the tree must +be updated with the new power numbers and weights. + +================ +3.2 Nomenclature +================ + + * dtpm_alloc() : Allocate and initialize a dtpm structure + + * dtpm_register() : Add the dtpm node to the tree + + * dtpm_register_parent() : Add an intermediate node + + * dtpm_unregister() : Remove the dtpm node from the tree + + * dtpm_update_power() : Update the power characteristics of the dtpm node
Signature · 1 line
2.17.1
Re: [PATCH v2 2/4] Documentation/powercap/dtpm: Add documentation for dtpm
Quoted 11 lines · On Mon, 16 Nov 2020 16:26:47 +0100 Daniel Lezcano <[email protected]>…
The dynamic thermal and power management is a technique to dynamically adjust the power consumption of different devices in order to ensure a global thermal constraint.
An userspace daemon is usually monitoring the temperature and the power to take immediate action on the device.
The DTPM framework provides an unified API to userspace to act on the power.
Document this framework.
It's always refreshing to see documentation show up with a new feature! :)
That said, it's clear that you haven't built the docs with this new material. There's a couple of little things I would ask you to do...
Quoted 5 lines
Signed-off-by Daniel Lezcano <[email protected]>--- Documentation/power/powercap/dtpm.rst | 222 ++++++++++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 Documentation/power/powercap/dtpm.rst
You also need to add this new file to index.rst
Quoted 25 lines
Documentation/power/powercap/dtpm.rst new file +19
-0@@ -0,0 +1,222 @@ +========================================== +Dynamic Thermal Power Management framework +========================================== + +On the embedded world, the complexity of the SoC leads to an +increasing number of hotspots which need to be monitored and mitigated +as a whole in order to prevent the temperature to go above the +normative and legally stated 'skin temperature'. + +Another aspect is to sustain the performance for a given power budget, +for example virtual reality where the user can feel dizziness if the +performance is capped while a big CPU is processing something else. Or +reduce the battery charging because the dissipated power is too high +compared with the power consumed by other devices. + +The userspace is the most adequate place to dynamically act on the +different devices by limiting their power given an application +profile: it has the knowledge of the platform. +
Documentation/power/powercap/dtpm.rst line 29
Quoted 10 lines
Documentation/power/powercap/dtpm.rst new file +10
-0+The Dynamic Thermal Power Management (DTPM) is a technique acting on +the device power by limiting and/or balancing a power budget among +different devices. + +The DTPM framework provides an unified interface to act on the +device power. + +=========== +1. Overview +===========
Please follow the sequence of subheading markers described in Documentation/doc-guide/sphinx.rst. This one should be:
Overview ========
(I took out the section number because that's not really needed in the RST world, but that's up to you).
Quoted 11 lines
Documentation/power/powercap/dtpm.rst new file +11
-0+The DTPM framework relies on the powercap framework to create the +powercap entries in the sysfs directory and implement the backend +driver to do the connection with the power manageable device. + +The DTPM is a tree representation describing the power constraints +shared between devices, not their physical positions. + +The nodes of the tree are a virtual description aggregating the power +characteristics of the children nodes and their power limitations. + +The leaves of the tree are the real power manageable devices.
Documentation/power/powercap/dtpm.rst line 51
Quoted 10 lines
Documentation/power/powercap/dtpm.rst new file +10
-0+ +For instance: + + SoC + | + `-- pkg + | + |-- pd0 (cpu0-3) + | + `-- pd1 (cpu4-5)
This formatting will throw errors when you try to do the docs build. The easiest solution is to put literal blocks like this into ... a literal block. And that is most easily done by putting a double colon at the end of the leading line:
For instance::
The same will need to happen for all the rest of the ascii art.
Documentation/power/powercap/dtpm.rst line 64
Quoted 12 lines
Documentation/power/powercap/dtpm.rst new file +12
-0+* The pkg power will be the sum of pd0 and pd1 power numbers. + + SoC (400mW - 3100mW) + | + `-- pkg (400mW - 3100mW) + | + |-- pd0 (100mW - 700mW) + | + `-- pd1 (300mW - 2400mW) + +* When the nodes are inserted in the tree, their power characteristics + are propagated to the parents.
I suspect you'll also need to indent the literal block another space or two to separate it from the bulleted list.
You don't have any kernel-doc directives to bring in the kerneldoc comments from the code. The code itself *does* have those comments, so you might want to add those.
Thanks,
jon
Re: [PATCH v2 2/4] Documentation/powercap/dtpm: Add documentation for dtpm
Quoted 13 lines · Jonathan Corbet
Quoted 10 lines · On Mon, 16 Nov 2020 16:26:47 +0100 Daniel Lezcano <[email protected]>…
The dynamic thermal and power management is a technique to dynamically adjust the power consumption of different devices in order to ensure a global thermal constraint.
An userspace daemon is usually monitoring the temperature and the power to take immediate action on the device.
The DTPM framework provides an unified API to userspace to act on the power.
Quoted 6 lines
Quoted 1 line
Document this framework.
It's always refreshing to see documentation show up with a new feature! :)
That said, it's clear that you haven't built the docs with this new material. There's a couple of little things I would ask you to do...
Thanks for the review, I'll take care of your comments and generate the documentation to check the expected result.
Signature · 5 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog
PATCH v2 3/4 powercap/drivers/dtpm: Add API for dynamic thermal power management
In reply to Daniel LezcanoOn the embedded world, the complexity of the SoC leads to an increasing number of hotspots which need to be monitored and mitigated as a whole in order to prevent the temperature to go above the normative and legally stated 'skin temperature'.
Another aspect is to sustain the performance for a given power budget, for example virtual reality where the user can feel dizziness if the GPU performance is capped while a big CPU is processing something else. Or reduce the battery charging because the dissipated power is too high compared with the power consumed by other devices.
The userspace is the most adequate place to dynamically act on the different devices by limiting their power given an application
profile: it has the knowledge of the platform.
These userspace daemons are in charge of the Dynamic Thermal Power Management (DTPM).
Nowadays, the dtpm daemons are abusing the thermal framework as they act on the cooling device state to force a specific and arbitrary state without taking care of the governor decisions. Given the closed loop of some governors that can confuse the logic or directly enter in a decision conflict.
As the number of cooling device support is limited today to the CPU and the GPU, the dtpm daemons have little control on the power dissipation of the system. The out of tree solutions are hacking around here and there in the drivers, in the frameworks to have control on the devices. The common solution is to declare them as cooling devices.
There is no unification of the power limitation unit, opaque states are used.
This patch provides a way to create a hierarchy of constraints using the powercap framework. The devices which are registered as power limit-able devices are represented in this hierarchy as a tree. They are linked together with intermediate nodes which are just there to propagate the constraint to the children.
The leaves of the tree are the real devices, the intermediate nodes are virtual, aggregating the children constraints and power characteristics.
Each node have a weight on a 2^10 basis, in order to reflect the percentage of power distribution of the children's node. This percentage is used to dispatch the power limit to the children.
The weight is computed against the max power of the siblings.
This simple approach allows to do a fair distribution of the power limit.
--- drivers/powercap/Kconfig | 6 + drivers/powercap/Makefile | 1 + drivers/powercap/dtpm.c | 436 ++++++++++++++++++++++++++++++ include/asm-generic/vmlinux.lds.h | 11 + include/linux/dtpm.h | 72 +++++ 5 files changed, 526 insertions(+) create mode 100644 drivers/powercap/dtpm.c create mode 100644 include/linux/dtpm.h
drivers/powercap/Kconfig +6 -0
@@ -43,4 +43,10 @@ config IDLE_INJECT CPUs for power capping. Idle period can be injected synchronously on a set of specified CPUs or alternatively on a per CPU basis. + +config DTPM + bool "Power capping for Dynamic Thermal Power Management" + help + This enables support for the power capping for the dynamic + thermal power management userspace engine. endif
drivers/powercap/Makefile +1 -0
@@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_DTPM) += dtpm.o obj-$(CONFIG_POWERCAP) += powercap_sys.o obj-$(CONFIG_INTEL_RAPL_CORE) += intel_rapl_common.o obj-$(CONFIG_INTEL_RAPL) += intel_rapl_msr.o
drivers/powercap/dtpm.c new file +436 -0
436 lines not on this page · raw message
include/asm-generic/vmlinux.lds.h +11 -0
@@ -316,6 +316,16 @@#define THERMAL_TABLE (name)#endif +#ifdef CONFIG_DTPM +#define DTPM_TABLE () \ + . = ALIGN(8); \ + __dtpm_table = .; \ + KEEP(*(__dtpm_table)) \ + __dtpm_table_end = .; +#else +#define DTPM_TABLE () +#endif +#define KERNEL_DTB () \ STRUCT_ALIGN(); \ __dtb_start = .; \
@@ -733,6 +743,7 @@ ACPI_PROBE_TABLE(irqchip) \ ACPI_PROBE_TABLE(timer) \ THERMAL_TABLE(governor) \ + DTPM_TABLE() \ EARLYCON_TABLE() \ LSM_TABLE() \ EARLY_LSM_TABLE() \
include/linux/dtpm.h new file +72 -0
72 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v2 4/4 powercap/drivers/dtpm: Add CPU energy model based support
In reply to Daniel LezcanoWith the powercap dtpm controller, we are able to plug devices with power limitation features in the tree.
The following patch introduces the CPU power limitation based on the energy model and the performance states.
The power limitation is done at the performance domain level. If some CPUs are unplugged, the corresponding power will be subtracted from the performance domain total power.
It is up to the platform to initialize the dtpm tree and add the CPU.
Here is an example to create a simple tree with one root node called "pkg" and the CPU's performance domains.
static int dtpm_register_pkg(struct dtpm_descr *descr)
{
struct dtpm *pkg;
int ret;
pkg = dtpm_alloc();
if (!pkg)
return -ENOMEM;
ret = dtpm_register_parent(descr->name, pkg, descr->parent);
if (ret)
return ret;
return dtpm_register_cpu(pkg);
}static struct dtpm_descr descr = {
.name = "pkg",
.init = dtpm_register_pkg,
};DTPM_DECLARE(descr);
--- drivers/powercap/Kconfig | 7 + drivers/powercap/Makefile | 1 + drivers/powercap/dtpm_cpu.c | 282 ++++++++++++++++++++++++++++++++++++ include/linux/cpuhotplug.h | 1 + include/linux/dtpm.h | 3 + 5 files changed, 294 insertions(+) create mode 100644 drivers/powercap/dtpm_cpu.c
drivers/powercap/Kconfig +7 -0
@@ -49,4 +49,11 @@ config DTPM help This enables support for the power capping for the dynamic thermal power management userspace engine. + +config DTPM_CPU + bool "Add CPU power capping based on the energy model" + depends on DTPM && ENERGY_MODEL + help + This enables support for CPU power limitation based on + energy model. endif
drivers/powercap/Makefile +1 -0
@@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_DTPM) += dtpm.o +obj-$(CONFIG_DTPM_CPU) += dtpm_cpu.o obj-$(CONFIG_POWERCAP) += powercap_sys.o obj-$(CONFIG_INTEL_RAPL_CORE) += intel_rapl_common.o obj-$(CONFIG_INTEL_RAPL) += intel_rapl_msr.o
drivers/powercap/dtpm_cpu.c new file +282 -0
282 lines not on this page · raw message
include/linux/cpuhotplug.h +1 -0
@@ -192,6 +192,7 @@ enum cpuhp_state {CPUHP_AP_ONLINE_DYN_END =CPUHP_AP_ONLINE_DYN +30 ,CPUHP_AP_X86_HPET_ONLINE ,CPUHP_AP_X86_KVM_CLK_ONLINE , +CPUHP_AP_DTPM_CPU_ONLINE ,CPUHP_AP_ACTIVE , CPUHP_ONLINE, };
include/linux/dtpm.h +3 -0
@@ -69,4 +69,7 @@ int dtpm_register_parent(const char *name, struct dtpm *dtpm,int dtpm_register (const char *name,struct dtpm *dtpm,struct dtpm *parent,struct powercap_zone_ops *ops,int nr_constraints,struct powercap_zone_constraint_ops *const_ops); + +int dtpm_register_cpu (struct dtpm *parent); +#endif
Signature · 1 line
2.17.1
Re: [PATCH v2 4/4] powercap/drivers/dtpm: Add CPU energy model based support
Hi Daniel,
Only one small comment regarding the setup of 'power_limit'.
Quoted 292 lines · Daniel Lezcano
With the powercap dtpm controller, we are able to plug devices with power limitation features in the tree.
The following patch introduces the CPU power limitation based on the energy model and the performance states.
The power limitation is done at the performance domain level. If some CPUs are unplugged, the corresponding power will be subtracted from the performance domain total power.
It is up to the platform to initialize the dtpm tree and add the CPU.
Here is an example to create a simple tree with one root node called "pkg" and the CPU's performance domains.
static int
dtpm_register_pkg(struct dtpm_descr *descr){ struct dtpm *pkg; int ret; pkg = dtpm_alloc(); if (!pkg) return -ENOMEM; ret = dtpm_register_parent(descr->name, pkg, descr->parent); if (ret) return ret; return dtpm_register_cpu(pkg); }static struct dtpm_descr descr = { .name = "pkg", .init = dtpm_register_pkg, };
DTPM_DECLARE(descr);Signed-off-by Daniel Lezcano <[email protected]>--- drivers/powercap/Kconfig | 7 + drivers/powercap/Makefile | 1 + drivers/powercap/dtpm_cpu.c | 282 ++++++++++++++++++++++++++++++++++++ include/linux/cpuhotplug.h | 1 + include/linux/dtpm.h | 3 + 5 files changed, 294 insertions(+)create mode 100644
drivers/powercap/dtpm_cpu.cdrivers/powercap/Kconfig +7
-0@@ -49,4 +49,11 @@ config DTPM help This enables support for the power capping for the dynamic thermal power management userspace engine. + +config DTPM_CPU + bool "Add CPU power capping based on the energy model" + depends on DTPM && ENERGY_MODEL + help + This enables support for CPU power limitation based on + energy model. endifdrivers/powercap/Makefile +1
-0@@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_DTPM) += dtpm.o +obj-$(CONFIG_DTPM_CPU) += dtpm_cpu.o obj-$(CONFIG_POWERCAP) += powercap_sys.o obj-$(CONFIG_INTEL_RAPL_CORE) += intel_rapl_common.o obj-$(CONFIG_INTEL_RAPL) += intel_rapl_msr.odrivers/powercap/dtpm_cpu.c new file +212
-0@@ -0,0 +1,282 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright 2020 Linaro Limited +* +* Author: Daniel Lezcano <[email protected]> +* +* The DTPM CPU is based on the energy model. It hooks the CPU in the +* DTPM tree which in turns update the power number by propagating the +* power number from the CPU energy model information to the parents. +* +* The association between the power and the performance state, allows +* to set the power of the CPU at the OPP granularity. +* +* The CPU hotplug is supported and the power numbers will be updated +* if a CPU is hot plugged / unplugged. +*/ +#include <linux/cpumask.h> +#include <linux/cpufreq.h> +#include <linux/cpuhotplug.h> +#include <linux/dtpm.h> +#include <linux/energy_model.h> +#include <linux/pm_qos.h> +#include <linux/slab.h> +#include <linux/units.h> + +static struct dtpm *__parent; + +static DEFINE_PER_CPU(struct dtpm *, dtpm_per_cpu); + +struct dtpm_cpu { +struct freq_qos_request qos_req; +int cpu; +}; + +/* +* When a new CPU is inserted at hotplug or boot time, add the power +* contribution and update the dtpm tree. +*/ +static int power_add (struct dtpm *dtpm,struct em_perf_domain *em) +{ +u64 power_min, power_max; + + power_min = em->table[0 ].power; + power_min *=MICROWATT_PER_MILLIWATT ; + power_min += dtpm->power_min; + + power_max = em->table[em->nr_perf_states -1 ].power; + power_max *=MICROWATT_PER_MILLIWATT ; + power_max += dtpm->power_max; + +return dtpm_update_power (dtpm, power_min, power_max); +} + +/* +* When a CPU is unplugged, remove its power contribution from the +* dtpm tree. +*/ +static int power_sub (struct dtpm *dtpm,struct em_perf_domain *em) +{ +u64 power_min, power_max; + + power_min = em->table[0 ].power; + power_min *=MICROWATT_PER_MILLIWATT ; + power_min = dtpm->power_min - power_min; + + power_max = em->table[em->nr_perf_states -1 ].power; + power_max *=MICROWATT_PER_MILLIWATT ; + power_max = dtpm->power_max - power_max; + +return dtpm_update_power (dtpm, power_min, power_max); +} + +static int set_pd_power_limit (struct powercap_zone *pcz,int cid, +u64 power_limit) +{ +struct dtpm *dtpm =to_dtpm (pcz); +struct dtpm_cpu *dtpm_cpu = dtpm->private; +struct em_perf_domain *pd; +struct cpumask cpus; +unsigned long freq; +u64 power; +int i, nr_cpus; + +spin_lock (&dtpm->lock); + + power_limit =clamp_val (power_limit, dtpm->power_min, dtpm->power_max); + + pd =em_cpu_get (dtpm_cpu->cpu); + +cpumask_and (&cpus, cpu_online_mask,to_cpumask (pd->cpus)); + + nr_cpus =cpumask_weight (&cpus); + +for (i =0 ; i < pd->nr_perf_states; i++) { + + power = pd->table[i].power *MICROWATT_PER_MILLIWATT * nr_cpus; + +if (power > power_limit) +break ; + } + + freq = pd->table[i -1 ].frequency; + + dtpm->power_limit = pd->table[i -1 ].power * +MICROWATT_PER_MILLIWATT * nr_cpus; + +spin_unlock (&dtpm->lock); + +freq_qos_update_request (&dtpm_cpu->qos_req, freq); + +return 0 ; +} + +static int get_pd_power_limit (struct powercap_zone *pcz,int cid,u64 *data) +{ +struct dtpm *dtpm =to_dtpm (pcz); + +spin_lock (&dtpm->lock); + *data = dtpm->power_limit; +spin_unlock (&dtpm->lock); + +return 0 ; +} + +static int get_pd_power_uw (struct powercap_zone *pcz,u64 *power_uw) +{ +struct dtpm *dtpm =to_dtpm (pcz); +struct dtpm_cpu *dtpm_cpu = dtpm->private; +struct em_perf_domain *pd; +unsigned long freq; +int i, nr_cpus; + + freq =cpufreq_quick_get (dtpm_cpu->cpu); + pd =em_cpu_get (dtpm_cpu->cpu); + nr_cpus =cpumask_weight (to_cpumask (pd->cpus)); + +for (i =0 ; i < pd->nr_perf_states; i++) { + +if (pd->table[i].frequency < freq) +continue ; + + *power_uw = pd->table[i].power * +MICROWATT_PER_MILLIWATT * nr_cpus; + +return 0 ; + } + +return -EINVAL ; +} + +static int cpu_release_zone (struct powercap_zone *pcz) +{ +struct dtpm *dtpm =to_dtpm (pcz); +struct dtpm_cpu *dtpm_cpu = dtpm->private; + +freq_qos_remove_request (&dtpm_cpu->qos_req); +kfree (dtpm_cpu); + +return dtpm_release_zone (pcz); +} + +static struct powercap_zone_constraint_ops pd_constraint_ops = { + .set_power_limit_uw = set_pd_power_limit, + .get_power_limit_uw = get_pd_power_limit, +}; + +static struct powercap_zone_ops pd_zone_ops = { + .get_power_uw = get_pd_power_uw, + .release = cpu_release_zone, +}; + +static int cpuhp_dtpm_cpu_offline (unsigned int cpu) +{ +struct cpufreq_policy *policy; +struct em_perf_domain *pd; +struct dtpm *dtpm; + + policy =cpufreq_cpu_get (cpu); + +if (!policy) +return 0 ; + + pd =em_cpu_get (cpu); +if (!pd) +return -EINVAL ; + + dtpm =per_cpu (dtpm_per_cpu, cpu); + +power_sub (dtpm, pd); + +if (cpumask_weight (policy->cpus) !=1 ) +return 0 ; + +for_each_cpu (cpu, policy->related_cpus) +per_cpu (dtpm_per_cpu, cpu) =NULL ; + +dtpm_unregister (dtpm); + +return 0 ; +} + +static int cpuhp_dtpm_cpu_online (unsigned int cpu) +{ +struct dtpm *dtpm; +struct dtpm_cpu *dtpm_cpu; +struct cpufreq_policy *policy; +struct em_perf_domain *pd; +char name[CPUFREQ_NAME_LEN ]; +int ret; + + policy =cpufreq_cpu_get (cpu); +
drivers/powercap/dtpm_cpu.c line 222
Quoted 10 lines
drivers/powercap/dtpm_cpu.c new file +10
-0+if (!policy) +return 0 ; + + pd =em_cpu_get (cpu); +if (!pd) +return -EINVAL ; + + dtpm =per_cpu (dtpm_per_cpu, cpu); +if (dtpm) +return power_add (dtpm, pd);
The dtpm->power_limit is not incremented in this path, when a new CPU joins the cluster.
Is it correct?
Or maybe we need something like:
------------------------------>8---------------------
if (dtpm) {
ret = power_add(dtpm, pd);
if (!ret)
dtpm->power_limit = dtpm->power_max;
return ret;
}------------------------8<---------------
The power_max should be updated after successful power_add(). It would disturb user set value in power_limit, though (described below).
Quoted 12 lines
drivers/powercap/dtpm_cpu.c new file +12
-0+ + dtpm =dtpm_alloc (); +if (!dtpm) +return -EINVAL ; + + dtpm_cpu =kzalloc (sizeof (dtpm_cpu),GFP_KERNEL ); +if (!dtpm_cpu) { +kfree (dtpm); +return -ENOMEM ; + } + + dtpm->private = dtpm_cpu;
drivers/powercap/dtpm_cpu.c line 244
Quoted 10 lines
drivers/powercap/dtpm_cpu.c new file +10
-0+ dtpm_cpu->cpu = cpu; + +for_each_cpu (cpu, policy->related_cpus) +per_cpu (dtpm_per_cpu, cpu) = dtpm; + + ret =power_add (dtpm, pd); +if (ret) + goto out_kfree_dtpm_cpu; + + dtpm->power_limit = dtpm->power_max;
Here, the power_limit will be set only once with power_max for a single CPU. I am not sure, but maybe we can simple say:
dtpm->power_limit = dtpm->power_max * cpumask_weight(policy->related_cpus)
an avoid touching it later (?)
Because this function can be called in runtime, when the power_limit was already set by userspace, the hotpluging in/out/in... CPU shouldn't change this limit.
Regards,
Lukasz
Re: [PATCH v2 4/4] powercap/drivers/dtpm: Add CPU energy model based support
Quoted 17 lines · Lukasz Luba
Hi Daniel,
Only one small comment regarding the setup of '
power_limit'.Quoted 12 lines · Daniel Lezcano
With the powercap dtpm controller, we are able to plug devices with power limitation features in the tree.
The following patch introduces the CPU power limitation based on the energy model and the performance states.
The power limitation is done at the performance domain level. If some CPUs are unplugged, the corresponding power will be subtracted from the performance domain total power.
It is up to the platform to initialize the dtpm tree and add the CPU.
[ ... ]
Quoted 8 lines
Quoted 4 lines
+
+ dtpm =per_cpu(dtpm_per_cpu, cpu);
+ if (dtpm)
+ returnpower_add(dtpm, pd);The
dtpm->power_limitis not incremented in this path, when a new CPU joins the cluster.
Is it correct?
Yes, you are right, there is something missing here. It does not change the behavior of the power capping, but the value will be inconsistent in the tree.
Quoted 41 lines
Or maybe we need something like:
------------------------------>8--------------------- if (dtpm) {
ret =power_add(dtpm, pd);
if (!ret)
dtpm->power_limit=dtpm->power_max; return ret;
} ------------------------8<---------------The
power_maxshould be updated after successfulpower_add(). It would disturb user set value inpower_limit, though (described below).Quoted 12 lines
drivers/powercap/dtpm_cpu.c new file +12
-0+ + dtpm = dtpm_alloc(); + if (!dtpm) + return -EINVAL; + + dtpm_cpu = kzalloc(sizeof(dtpm_cpu), GFP_KERNEL); + if (!dtpm_cpu) { + kfree(dtpm); + return -ENOMEM; + } + + dtpm->private = dtpm_cpu;drivers/powercap/dtpm_cpu.c line 244
Quoted 10 lines
drivers/powercap/dtpm_cpu.c new file +10
-0+ dtpm_cpu->cpu = cpu; + + for_each_cpu(cpu, policy->related_cpus) + per_cpu(dtpm_per_cpu, cpu) = dtpm; + + ret = power_add(dtpm, pd); + if (ret) + goto out_kfree_dtpm_cpu; + + dtpm->power_limit = dtpm->power_max;Here, the
power_limitwill be set only once withpower_maxfor a single CPU. I am not sure, but maybe we can simple say:
Quoted 7 lines
dtpm->power_limit=dtpm->power_max*cpumask_weight(policy->related_cpus)an avoid touching it later (?)
Because this function can be called in runtime, when the
power_limitwas already set by userspace, the hotpluging in/out/in... CPU shouldn't change this limit.
Hmm, I have to think about it because the power_limit is always less or equal to power_max.
Signature · 5 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog