Skip to contentSkip to conversation

Subject: The power allocator thermal governor

PATCH In review Javi Merino 5 participants 21 messages Mar 2, 2015 – Mar 15, 2017

Subsystems (4)

v3 of 6 In review Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover Subject: The power allocator thermal governor

*** BLURB HERE ***

Hi linux-pm,

Introduce the power allocator governor, a thermal governor that allocates device power to control temperature. This series is based on branch "linus" of Eduardo's linux-soc-thermal tree:

git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal.git

Changes since v2:
- Address Eduardo's review
+ Turn variable-size array in divvy_up_power() into a devm_kcalloc() as suggested by Eduardo
+ Remove #ifdeffery from thermal_core.c as suggested by Eduardo
- Bring back cpufreq's CPUFREQ_UPDATE_POLICY_CPU notifier in order to update the cpu device in cpu_cooling.c when cpufreq changes the policy cpu.

Changes since v1:
- Removed a memory leak pointed out by Steve Rostedt
- Relax the constraint of the trip points to be the first and last passive trip point as Lina Iyer suggested.
- Update how the governor treats weights to mimic the behavior of fair share in v3 of the weight fixes series.
- Don't cache the cpufreq_cooling_device and scan for it whenever we need it.
- Consider the rate of changes in the derivative term of the PID controller

Changes since RFC v6:
- Addressed Eduardo's review
+ Pass the interval to the static power function as suggested by Eduardo
+ Make the cooling device ops return 0 or -E* and put the calculation in a parameter, like the rest of the cooling device ops
+ Documentation improvements
- Use thermal_cdev_update() to change cooling device states
- Add a patch to export the power allocator governor's tzp parameters to sysfs

Changes since RFC v5:
- Addressed Stephen's review of the trace patches.
- Removed power actors and extended the cooling device interface instead.
- Let platforms override the power allocator governor parameters in their thermal zone parameters

Changes since RFC v4:
- Add more tracing
- Document some of the limitations of the power allocator governor
- Export the power_actor API and move power_actor.h to include/linux

Changes since RFC v3:
- Use tz->passive to poll faster when the first trip point is hit.
- Don't make a special directory for power_actors
- Add a DT property for sustainable-power
- Simplify the static power interface and pass the current thermal zone in every power_actor_ops to remove the controversial enum power_actor_types
- Use locks with the actor_list list
- Use cpufreq_get() to get the frequency of the cpu instead of using the notifiers.
- Remove the prompt for THERMAL_POWER_ACTOR_CPU when configuring the kernel

Changes since RFC v2:
- Changed the PI controller into a PID controller
- Added static power to the cpu power model
- tz parameter max_dissipatable_power renamed to sustainable_power
- Register the cpufreq cooling device as part of the power_cpu_actor registration.

Changes since RFC v1:
- Fixed finding cpufreq cooling devices in cpufreq_frequency_change()
- Replaced the cooling device interface with a separate power actor API
- Addressed most of Eduardo's comments
- Incorporated ftrace support for bitmask to trace cpumasks

Cheers,
Javi & Punit

Javi Merino (4):
thermal: introduce the Power Allocator governor
thermal: add trace events to the power allocator governor
thermal: export thermal_zone_parameters to sysfs Revert "cpufreq: remove CPUFREQ_UPDATE_POLICY_CPU notifications"

Kapileshwar Singh (1):
thermal: cpu_cooling: update the cpu device when cpufreq updates the
policy cpu

 Documentation/thermal/power_allocator.txt      | 247 ++++++++++++
 Documentation/thermal/sysfs-api.txt            |  52 +++
 drivers/cpufreq/cpufreq.c                      |   3 +
 drivers/thermal/Kconfig                        |  15 +
 drivers/thermal/Makefile                       |   1 +
 drivers/thermal/cpu_cooling.c                  |  34 +-
 drivers/thermal/power_allocator.c              | 538 +++++++++++++++++++++++++
 drivers/thermal/thermal_core.c                 | 110 ++++-
 drivers/thermal/thermal_core.h                 |   8 +
 include/linux/cpufreq.h                        |   5 +-
 include/linux/thermal.h                        |  37 +-
 include/trace/events/thermal.h                 |  58 +++
 include/trace/events/thermal_power_allocator.h |  87 ++++
 13 files changed, 1184 insertions(+), 11 deletions(-)
 create mode 100644 Documentation/thermal/power_allocator.txt
 create mode 100644 drivers/thermal/power_allocator.c
 create mode 100644 include/trace/events/thermal_power_allocator.h
Signature · 1 line
1.9.1
#1 lore raw

PATCH v3 1/5 thermal: introduce the Power Allocator governor

The power allocator governor is a thermal governor that controls system and device power allocation to control temperature. Conceptually, the implementation divides the sustainable power of a thermal zone among all the heat sources in that zone.

This governor relies on "power actors", entities that represent heat sources. They can report current and maximum power consumption and can set a given maximum power consumption, usually via a cooling device.

The governor uses a Proportional Integral Derivative (PID) controller driven by the temperature of the thermal zone. The output of the controller is a power budget that is then allocated to each power actor that can have bearing on the temperature we are trying to control. It decides how much power to give each cooling device based on the performance they are requesting. The PID controller ensures that the total power budget does not exceed the control temperature.

---
 Documentation/thermal/power_allocator.txt | 247 ++++++++++++++
 drivers/thermal/Kconfig                   |  15 +
 drivers/thermal/Makefile                  |   1 +
 drivers/thermal/power_allocator.c         | 520 ++++++++++++++++++++++++++++++
 drivers/thermal/thermal_core.c            |   9 +-
 drivers/thermal/thermal_core.h            |   8 +
 include/linux/thermal.h                   |  37 ++-
 7 files changed, 830 insertions(+), 7 deletions(-)
 create mode 100644 Documentation/thermal/power_allocator.txt
 create mode 100644 drivers/thermal/power_allocator.c
Documentation/thermal/power_allocator.txt new file +247 -0
new file mode 100644
index 000000000000..c3797b529991
@@ -0,0 +1,247 @@
+Power allocator governor tunables
+=================================
+
+Trip points
+-----------
+
+The governor requires the following two passive trip points:
+
+1.  "switch on" trip point: temperature above which the governor
+    control loop starts operating.  This is the first passive trip
+    point of the thermal zone.
+
+2.  "desired temperature" trip point: it should be higher than the
+    "switch on" trip point.  This the target temperature the governor
+    is controlling for.  This is the last passive trip point of the
+    thermal zone.
+
+PID Controller
+--------------
+
+The power allocator governor implements a
+Proportional-Integral-Derivative controller (PID controller) with
+temperature as the control input and power as the controlled output:
+
+    P_max = k_p * e + k_i * err_integral + k_d * diff_err + sustainable_power
+
+where
+    e = desired_temperature - current_temperature
+    err_integral is the sum of previous errors
+    diff_err = e - previous_error
+
+It is similar to the one depicted below:
+
+                                      k_d
+                                       |
+current_temp                           |
+     |                                 v
+     |                +----------+   +---+
+     |         +----->| diff_err |-->| X |------+
+     |         |      +----------+   +---+      |
+     |         |                                |      tdp        actor
+     |         |                      k_i       |       |  get_requested_power()
+     |         |                       |        |       |        |     |
+     |         |                       |        |       |        |     | ...
+     v         |                       v        v       v        v     v
+   +---+       |      +-------+      +---+    +---+   +---+   +----------+
+   | S |-------+----->| sum e |----->| X |--->| S |-->| S |-->|power     |
+   +---+       |      +-------+      +---+    +---+   +---+   |allocation|
+     ^         |                                ^             +----------+
+     |         |                                |                |     |
+     |         |        +---+                   |                |     |
+     |         +------->| X |-------------------+                v     v
+     |                  +---+                               granted performance
+desired_temperature       ^
+                          |
+                          |
+                      k_po/k_pu
+
+Sustainable power
+-----------------
+
+An estimate of the sustainable dissipatable power (in mW) should be
+provided while registering the thermal zone.  This estimates the
+sustained power that can be dissipated at the desired control
+temperature.  This is the maximum sustained power for allocation at
+the desired maximum temperature.  The actual sustained power can vary
+for a number of reasons.  The closed loop controller will take care of
+variations such as environmental conditions, and some factors related
+to the speed-grade of the silicon.  `sustainable_power` is therefore
+simply an estimate, and may be tuned to affect the aggressiveness of
+the thermal ramp. For reference, the sustainable power of a 4" phone
+is typically 2000mW, while on a 10" tablet is around 4500mW (may vary
+depending on screen size).
+
+If you are using device tree, do add it as a property of the
+thermal-zone.  For example:
+
+	thermal-zones {
+		soc_thermal {
+			polling-delay = <1000>;
+			polling-delay-passive = <100>;
+			sustainable-power = <2500>;
+			...
+
+Instead, if the thermal zone is registered from the platform code, pass a
+`thermal_zone_params` that has a `sustainable_power`.  If no
+`thermal_zone_params` were being passed, then something like below
+will suffice:
+
+	static const struct thermal_zone_params tz_params = {
+		.sustainable_power = 3500,
+	};
+
+and then pass `tz_params` as the 5th parameter to
+`thermal_zone_device_register()`
+
+k_po and k_pu
+-------------
+
+The implementation of the PID controller in the power allocator
+thermal governor allows the configuration of two proportional term
+constants: `k_po` and `k_pu`.  `k_po` is the proportional term
+constant during temperature overshoot periods (current temperature is
+above "desired temperature" trip point).  Conversely, `k_pu` is the
+proportional term constant during temperature undershoot periods
+(current temperature below "desired temperature" trip point).
+
+These controls are intended as the primary mechanism for configuring
+the permitted thermal "ramp" of the system.  For instance, a lower
+`k_pu` value will provide a slower ramp, at the cost of capping
+available capacity at a low temperature.  On the other hand, a high
+value of `k_pu` will result in the governor granting very high power
+whilst temperature is low, and may lead to temperature overshooting.
+
+The default value for `k_pu` is:
+
+    2 * sustainable_power / (desired_temperature - switch_on_temp)
+
+This means that at `switch_on_temp` the output of the controller's
+proportional term will be 2 * `sustainable_power`.  The default value
+for `k_po` is:
+
+    sustainable_power / (desired_temperature - switch_on_temp)
+
+Focusing on the proportional and feed forward values of the PID
+controller equation we have:
+
+    P_max = k_p * e + sustainable_power
+
+The proportional term is proportional to the difference between the
+desired temperature and the current one.  When the current temperature
+is the desired one, then the proportional component is zero and
+`P_max` = `sustainable_power`.  That is, the system should operate in
+thermal equilibrium under constant load.  `sustainable_power` is only
+an estimate, which is the reason for closed-loop control such as this.
+
+Expanding `k_pu` we get:
+    P_max = 2 * sustainable_power * (T_set - T) / (T_set - T_on) +
+        sustainable_power
+
+where
+    T_set is the desired temperature
+    T is the current temperature
+    T_on is the switch on temperature
+
+When the current temperature is the switch_on temperature, the above
+formula becomes:
+
+    P_max = 2 * sustainable_power * (T_set - T_on) / (T_set - T_on) +
+        sustainable_power = 2 * sustainable_power + sustainable_power =
+        3 * sustainable_power
+
+Therefore, the proportional term alone linearly decreases power from
+3 * `sustainable_power` to `sustainable_power` as the temperature
+rises from the switch on temperature to the desired temperature.
+
+k_i and integral_cutoff
+-----------------------
+
+`k_i` configures the PID loop's integral term constant.  This term
+allows the PID controller to compensate for long term drift and for
+the quantized nature of the output control: cooling devices can't set
+the exact power that the governor requests.  When the temperature
+error is below `integral_cutoff`, errors are accumulated in the
+integral term.  This term is then multiplied by `k_i` and the result
+added to the output of the controller.  Typically `k_i` is set low (1
+or 2) and `integral_cutoff` is 0.
+
+k_d
+---
+
+`k_d` configures the PID loop's derivative term constant.  It's
+recommended to leave it as the default: 0.
+
+Cooling device power API
+========================
+
+Cooling devices controlled by this governor must supply the additional
+"power" API in their `cooling_device_ops`.  It consists on three ops:
+
+1. int get_requested_power(struct thermal_cooling_device *cdev,
+	struct thermal_zone_device *tz, u32 *power);
+@cdev: The `struct thermal_cooling_device` pointer
+@tz: thermal zone in which we are currently operating
+@power: pointer in which to store the calculated power
+
+`get_requested_power()` calculates the power requested by the device
+in milliwatts and stores it in @power .  It should return 0 on
+success, -E* on failure.  This is currently used by the power
+allocator governor to calculate how much power to give to each cooling
+device.
+
+2. int state2power(struct thermal_cooling_device *cdev, struct
+        thermal_zone_device *tz, unsigned long state, u32 *power);
+@cdev: The `struct thermal_cooling_device` pointer
+@tz: thermal zone in which we are currently operating
+@state: A cooling device state
+@power: pointer in which to store the equivalent power
+
+Convert cooling device state @state into power consumption in
+milliwatts and store it in @power.  It should return 0 on success, -E*
+on failure.  This is currently used by thermal core to calculate the
+maximum power that an actor can consume.
+
+3. int power2state(struct thermal_cooling_device *cdev, u32 power,
+	unsigned long *state);
+@cdev: The `struct thermal_cooling_device` pointer
+@power: power in milliwatts
+@state: pointer in which to store the resulting state
+
+Calculate a cooling device state that would make the device consume at
+most @power mW and store it in @state.  It should return 0 on success,
+-E* on failure.  This is currently used by the thermal core to convert
+a given power set by the power allocator governor to a state that the
+cooling device can set.  It is a function because this conversion may
+depend on external factors that may change so this function should the
+best conversion given "current circumstances".
+
+Cooling device weights
+----------------------
+
+Weights are a mechanism to bias the allocation among cooling
+devices.  They express the relative power efficiency of different
+cooling devices.  Higher weight can be used to express higher power
+efficiency.  Weighting is relative such that if each cooling device
+has a weight of one they are considered equal.  This is particularly
+useful in heterogeneous systems where two cooling devices may perform
+the same kind of compute, but with different efficiency.  For example,
+a system with two different types of processors.
+
+If the thermal zone is registered using
+`thermal_zone_device_register()` (i.e., platform code), then weights
+are passed as part of the thermal zone's `thermal_bind_parameters`.
+If the platform is registered using device tree, then they are passed
+as the `contribution` property of each map in the `cooling-maps` node.
+
+Limitations of the power allocator governor
+===========================================
+
+The power allocator governor's PID controller works best if there is a
+periodic tick.  If you have a driver that calls
+`thermal_zone_device_update()` (or anything that ends up calling the
+governor's `throttle()` function) repetitively, the governor response
+won't be very good.  Note that this is not particular to this
+governor, step-wise will also misbehave if you call its throttle()
+faster than the normal thermal framework tick (due to interrupts for
+example) as it will overreact.
drivers/thermal/Kconfig +15 -0
index 30aee81e9f5b..a1b43eab0a70 100644
@@ -71,6 +71,14 @@ config THERMAL_DEFAULT_GOV_USER_SPACE
 	  Select this if you want to let the user space manage the
 	  platform thermals.
 
+config THERMAL_DEFAULT_GOV_POWER_ALLOCATOR
+	bool "power_allocator"
+	select THERMAL_GOV_POWER_ALLOCATOR
+	help
+	  Select this if you want to control temperature based on
+	  system and device power allocation. This governor can only
+	  operate on cooling devices that implement the power API.
+
 endchoice
 
 config THERMAL_GOV_FAIR_SHARE
@@ -99,6 +107,13 @@ config THERMAL_GOV_USER_SPACE
 	help
 	  Enable this to let the user space manage the platform thermals.
 
+config THERMAL_GOV_POWER_ALLOCATOR
+	bool "Power allocator thermal governor"
+	select THERMAL_POWER_ACTOR
+	help
+	  Enable this to manage platform thermals by dynamically
+	  allocating and limiting power to devices.
+
 config CPU_THERMAL
 	bool "generic cpu cooling support"
 	depends on CPU_FREQ
drivers/thermal/Makefile +1 -0
index 1fe86652cfb6..b1783cf37ed2 100644
@@ -14,6 +14,7 @@ thermal_sys-$(CONFIG_THERMAL_GOV_FAIR_SHARE)	+= fair_share.o
 thermal_sys-$(CONFIG_THERMAL_GOV_BANG_BANG)	+= gov_bang_bang.o
 thermal_sys-$(CONFIG_THERMAL_GOV_STEP_WISE)	+= step_wise.o
 thermal_sys-$(CONFIG_THERMAL_GOV_USER_SPACE)	+= user_space.o
+thermal_sys-$(CONFIG_THERMAL_GOV_POWER_ALLOCATOR)	+= power_allocator.o
 
 # cpufreq cooling
 thermal_sys-$(CONFIG_CPU_THERMAL)	+= cpu_cooling.o
drivers/thermal/power_allocator.c new file +520 -0

520 lines not on this page · raw message

drivers/thermal/thermal_core.c +7 -2
index 82f1cca9c8ce..df9ba3bf55dc 100644
@@ -1617,7 +1617,7 @@ static void remove_trip_attrs(struct thermal_zone_device *tz)
 struct thermal_zone_device *thermal_zone_device_register(const char *type,
 	int trips, int mask, void *devdata,
 	struct thermal_zone_device_ops *ops,
-	const struct thermal_zone_params *tzp,
+	struct thermal_zone_params *tzp,
 	int passive_delay, int polling_delay)
 {
 	struct thermal_zone_device *tz;
@@ -1969,7 +1969,11 @@ static int __init thermal_register_governors(void)
 	if (result)
 		return result;
 
-	return thermal_gov_user_space_register();
+	result = thermal_gov_user_space_register();
+	if (result)
+		return result;
+
+	return thermal_gov_power_allocator_register();
 }
 
 static void thermal_unregister_governors(void)
@@ -1978,6 +1982,7 @@ static void thermal_unregister_governors(void)
 	thermal_gov_fair_share_unregister();
 	thermal_gov_bang_bang_unregister();
 	thermal_gov_user_space_unregister();
+	thermal_gov_power_allocator_unregister();
 }
 
 static int __init thermal_init(void)
drivers/thermal/thermal_core.h +8 -0
index faebe881f062..8a6624488cc5 100644
@@ -88,6 +88,14 @@ static inline int thermal_gov_user_space_register(void) { return 0; }
 static inline void thermal_gov_user_space_unregister(void) {}
 #endif /* CONFIG_THERMAL_GOV_USER_SPACE */
 
+#ifdef CONFIG_THERMAL_GOV_POWER_ALLOCATOR
+int thermal_gov_power_allocator_register(void);
+void thermal_gov_power_allocator_unregister(void);
+#else
+static inline int thermal_gov_power_allocator_register(void) { return 0; }
+static inline void thermal_gov_power_allocator_unregister(void) {}
+#endif /* CONFIG_THERMAL_GOV_POWER_ALLOCATOR */
+
 /* device tree support */
 #ifdef CONFIG_THERMAL_OF
 int of_parse_thermal_zones(void);
include/linux/thermal.h +32 -5
index d69b1806f430..f8f503cd50a6 100644
@@ -59,6 +59,8 @@
 #define DEFAULT_THERMAL_GOVERNOR       "fair_share"
 #elif defined(CONFIG_THERMAL_DEFAULT_GOV_USER_SPACE)
 #define DEFAULT_THERMAL_GOVERNOR       "user_space"
+#elif defined(CONFIG_THERMAL_DEFAULT_GOV_POWER_ALLOCATOR)
+#define DEFAULT_THERMAL_GOVERNOR       "power_allocator"
 #endif
 
 struct thermal_zone_device;
@@ -154,8 +156,7 @@ struct thermal_attr {
  * @devdata:	private pointer for device private data
  * @trips:	number of trip points the thermal zone supports
  * @passive_delay:	number of milliseconds to wait between polls when
- *			performing passive cooling.  Currenty only used by the
- *			step-wise governor
+ *			performing passive cooling.
  * @polling_delay:	number of milliseconds to wait between polls when
  *			checking whether trip points have been crossed (0 for
  *			interrupt driven systems)
@@ -165,7 +166,6 @@ struct thermal_attr {
  * @last_temperature:	previous temperature read
  * @emul_temperature:	emulated temperature when using CONFIG_THERMAL_EMULATION
  * @passive:		1 if you've crossed a passive trip point, 0 otherwise.
- *			Currenty only used by the step-wise governor.
  * @forced_passive:	If > 0, temperature at which to switch on all ACPI
  *			processor cooling devices.  Currently only used by the
  *			step-wise governor.
@@ -197,7 +197,7 @@ struct thermal_zone_device {
 	int passive;
 	unsigned int forced_passive;
 	struct thermal_zone_device_ops *ops;
-	const struct thermal_zone_params *tzp;
+	struct thermal_zone_params *tzp;
 	struct thermal_governor *governor;
 	void *governor_data;
 	struct list_head thermal_instances;
@@ -275,6 +275,33 @@ struct thermal_zone_params {
 
 	int num_tbps;	/* Number of tbp entries */
 	struct thermal_bind_params *tbp;
+
+	/*
+	 * Sustainable power (heat) that this thermal zone can dissipate in
+	 * mW
+	 */
+	u32 sustainable_power;
+
+	/*
+	 * Proportional parameter of the PID controller when
+	 * overshooting (i.e., when temperature is below the target)
+	 */
+	s32 k_po;
+
+	/*
+	 * Proportional parameter of the PID controller when
+	 * undershooting
+	 */
+	s32 k_pu;
+
+	/* Integral parameter of the PID controller */
+	s32 k_i;
+
+	/* Derivative parameter of the PID controller */
+	s32 k_d;
+
+	/* threshold below which the error is no longer accumulated */
+	s32 integral_cutoff;
 };
 
 struct thermal_genl_event {
@@ -349,7 +376,7 @@ int power_actor_set_power(struct thermal_cooling_device *,
 			  struct thermal_instance *, u32);
 struct thermal_zone_device *thermal_zone_device_register(const char *, int, int,
 		void *, struct thermal_zone_device_ops *,
-		const struct thermal_zone_params *, int, int);
+		struct thermal_zone_params *, int, int);
 void thermal_zone_device_unregister(struct thermal_zone_device *);
 
 int thermal_zone_bind_cooling_device(struct thermal_zone_device *, int,
Signature · 1 line
1.9.1
#2 lore raw

PATCH v3 2/5 thermal: add trace events to the power allocator governor

In reply to Javi Merino

Add trace events for the power allocator governor and the power actor interface of the cpu cooling device.

Cc Zhang Rui <[email protected]>
Cc Eduardo Valentin <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 drivers/thermal/cpu_cooling.c                  | 31 ++++++++-
 drivers/thermal/power_allocator.c              | 22 ++++++-
 include/trace/events/thermal.h                 | 58 +++++++++++++++++
 include/trace/events/thermal_power_allocator.h | 87 ++++++++++++++++++++++++++
 4 files changed, 194 insertions(+), 4 deletions(-)
 create mode 100644 include/trace/events/thermal_power_allocator.h
drivers/thermal/cpu_cooling.c +29 -2
index ba23150c7bde..c4974144c787 100644
@@ -31,6 +31,8 @@
 #include <linux/cpu.h>
 #include <linux/cpu_cooling.h>
 
+#include <trace/events/thermal.h>
+
 /*
  * Cooling state <-> CPUFreq frequency
  *
@@ -588,12 +590,20 @@ static int cpufreq_get_requested_power(struct thermal_cooling_device *cdev,
 				       u32 *power)
 {
 	unsigned long freq;
-	int cpu, ret;
+	int i = 0, cpu, ret;
 	u32 static_power, dynamic_power, total_load = 0;
 	struct cpufreq_cooling_device *cpufreq_device = cdev->devdata;
+	u32 *load_cpu = NULL;
 
 	freq = cpufreq_quick_get(cpumask_any(&cpufreq_device->allowed_cpus));
 
+	if (trace_thermal_power_cpu_get_power_enabled()) {
+		u32 ncpus = cpumask_weight(&cpufreq_device->allowed_cpus);
+
+		load_cpu = devm_kcalloc(&cdev->device, ncpus, sizeof(*load_cpu),
+					GFP_KERNEL);
+	}
+
 	for_each_cpu(cpu, &cpufreq_device->allowed_cpus) {
 		u32 load;
 
@@ -603,14 +613,29 @@ static int cpufreq_get_requested_power(struct thermal_cooling_device *cdev,
 			load = 0;
 
 		total_load += load;
+		if (trace_thermal_power_cpu_limit_enabled() && load_cpu)
+			load_cpu[i] = load;
+
+		i++;
 	}
 
 	cpufreq_device->last_load = total_load;
 
 	dynamic_power = get_dynamic_power(cpufreq_device, freq);
 	ret = get_static_power(cpufreq_device, tz, freq, &static_power);
-	if (ret)
+	if (ret) {
+		if (load_cpu)
+			devm_kfree(&cdev->device, load_cpu);
 		return ret;
+	}
+
+	if (load_cpu) {
+		trace_thermal_power_cpu_get_power(
+			&cpufreq_device->allowed_cpus,
+			freq, load_cpu, i, dynamic_power, static_power);
+
+		devm_kfree(&cdev->device, load_cpu);
+	}
 
 	*power = static_power + dynamic_power;
 	return 0;
@@ -718,6 +743,8 @@ static int cpufreq_power2state(struct thermal_cooling_device *cdev,
 		return -EINVAL;
 	}
 
+	trace_thermal_power_cpu_limit(&cpufreq_device->allowed_cpus,
+				      target_freq, *state, power);
 	return 0;
 }
 
drivers/thermal/power_allocator.c +20 -2
index 67982d79b76c..3ca7530deac6 100644
@@ -19,6 +19,9 @@
 #include <linux/slab.h>
 #include <linux/thermal.h>
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/thermal_power_allocator.h>
+
 #include "thermal_core.h"
 
 #define FRAC_BITS 10
@@ -138,7 +141,14 @@ static u32 pid_controller(struct thermal_zone_device *tz,
 	/* feed-forward the known sustainable dissipatable power */
 	power_range = tz->tzp->sustainable_power + frac_to_int(power_range);
 
-	return clamp(power_range, (s64)0, (s64)max_allocatable_power);
+	power_range = clamp(power_range, (s64)0, (s64)max_allocatable_power);
+
+	trace_thermal_power_allocator_pid(tz, frac_to_int(err),
+					  frac_to_int(params->err_integral),
+					  frac_to_int(p), frac_to_int(i),
+					  frac_to_int(d), power_range);
+
+	return power_range;
 }
 
 /**
@@ -219,7 +229,7 @@ static int allocate_power(struct thermal_zone_device *tz,
 	struct power_allocator_params *params = tz->governor_data;
 	u32 *req_power, *max_power, *granted_power, *extra_actor_power;
 	u32 total_req_power, max_allocatable_power;
-	u32 power_range;
+	u32 total_granted_power, power_range;
 	int i, num_actors, total_weight, ret = 0;
 	int trip_max_desired_temperature = params->trip_max_desired_temperature;
 
@@ -295,6 +305,7 @@ static int allocate_power(struct thermal_zone_device *tz,
 	divvy_up_power(req_power, max_power, num_actors, total_req_power,
 		       power_range, granted_power, extra_actor_power);
 
+	total_granted_power = 0;
 	i = 0;
 	list_for_each_entry(instance, &tz->thermal_instances, tz_node) {
 		if (instance->trip != trip_max_desired_temperature)
@@ -305,10 +316,17 @@ static int allocate_power(struct thermal_zone_device *tz,
 
 		power_actor_set_power(instance->cdev, instance,
 				      granted_power[i]);
+		total_granted_power += granted_power[i];
 
 		i++;
 	}
 
+	trace_thermal_power_allocator(tz, req_power, total_req_power,
+				      granted_power, total_granted_power,
+				      num_actors, power_range,
+				      max_allocatable_power, current_temp,
+				      (s32)control_temp - (s32)current_temp);
+
 	devm_kfree(&tz->device, req_power);
 unlock:
 	mutex_unlock(&tz->lock);
include/trace/events/thermal.h +58 -0

64 lines not on this page · raw message

include/trace/events/thermal_power_allocator.h new file +87 -0

87 lines not on this page · raw message

Signature · 1 line
1.9.1
maintainer
#3 lore raw

Re: [PATCH v3 2/5] thermal: add trace events to the power allocator governor

Hi Javi,

Sorry for picking up an old thread, but i had a question for you.

Quoted 5 lines · Javi Merino
drivers/thermal/cpu_cooling.c +0 -0
@@ -588,12 +590,20 @@ static int cpufreq_get_requested_power(struct thermal_cooling_device *cdev,
                                       u32 *power)
 {

drivers/thermal/cpu_cooling.c static int cpufreq_get_requested_power(struct thermal_cooling_device *cdev, · line 600

Quoted 10 lines
drivers/thermal/cpu_cooling.c +3 -1
        unsigned long freq;
-       int cpu, ret;
+       int i = 0, cpu, ret;
        u32 static_power, dynamic_power, total_load = 0;
        struct cpufreq_cooling_device *cpufreq_device = cdev->devdata;
+       u32 *load_cpu = NULL;
 
        freq = cpufreq_quick_get(cpumask_any(&cpufreq_device->allowed_cpus));
 
+       if (trace_thermal_power_cpu_get_power_enabled()) {

You allocated load_cpu if cpu_get_power trace is enabled.

Quoted 10 lines
drivers/thermal/cpu_cooling.c +6 -0
+               u32 ncpus = cpumask_weight(&cpufreq_device->allowed_cpus);
+
+               load_cpu = devm_kcalloc(&cdev->device, ncpus, sizeof(*load_cpu),
+                                       GFP_KERNEL);
+       }
+
        for_each_cpu(cpu, &cpufreq_device->allowed_cpus) {
                u32 load;
 

drivers/thermal/cpu_cooling.c static int cpufreq_get_requested_power(struct thermal_cooling_device *cdev, · line 616

Quoted 4 lines
drivers/thermal/cpu_cooling.c +1 -0
@@ -603,14 +613,29 @@ static int cpufreq_get_requested_power(struct thermal_cooling_device *cdev,
                        load = 0;
 
                total_load += load;
+               if (trace_thermal_power_cpu_limit_enabled() && load_cpu)

But you store values to it only if cpu_limit trace is also enabled, otherwise it is all zeros.

Why?

#4 lore raw

PATCH v3 3/5 thermal: export thermal_zone_parameters to sysfs

In reply to Javi Merino

It's useful for tuning to be able to edit thermal_zone_parameters from userspace. Export them to the thermal_zone sysfs so that they can be easily changed.

Cc Zhang Rui <[email protected]>
Cc Eduardo Valentin <[email protected]>
---
 Documentation/thermal/sysfs-api.txt |  52 +++++++++++++++++++
 drivers/thermal/thermal_core.c      | 101 ++++++++++++++++++++++++++++++++++++
 2 files changed, 153 insertions(+)
Documentation/thermal/sysfs-api.txt +52 -0
index fc7dfe10778b..7d44d7f1a71b 100644
@@ -184,6 +184,12 @@ Thermal zone device sys I/F, created once it's registered:
     |---trip_point_[0-*]_type:	Trip point type
     |---trip_point_[0-*]_hyst:	Hysteresis value for this trip point
     |---emul_temp:		Emulated temperature set node
+    |---sustainable_power:      Sustainable dissipatable power
+    |---k_po:                   Proportional term during temperature overshoot
+    |---k_pu:                   Proportional term during temperature undershoot
+    |---k_i:                    PID's integral term in the power allocator gov
+    |---k_d:                    PID's derivative term in the power allocator
+    |---integral_cutoff:        Offset above which errors are accumulated
 
 Thermal cooling device sys I/F, created once it's registered:
 /sys/class/thermal/cooling_device[0-*]:
@@ -307,6 +313,52 @@ emul_temp
 	  because userland can easily disable the thermal policy by simply
 	  flooding this sysfs node with low temperature values.
 
+sustainable_power
+	An estimate of the sustained power that can be dissipated by
+	the thermal zone. Used by the power allocator governor. For
+	more information see Documentation/thermal/power_allocator.txt
+	Unit: milliwatts
+	RW, Optional
+
+k_po
+	The proportional term of the power allocator governor's PID
+	controller during temperature overshoot. Temperature overshoot
+	is when the current temperature is above the "desired
+	temperature" trip point. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
+k_pu
+	The proportional term of the power allocator governor's PID
+	controller during temperature undershoot. Temperature undershoot
+	is when the current temperature is below the "desired
+	temperature" trip point. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
+k_i
+	The integral term of the power allocator governor's PID
+	controller. This term allows the PID controller to compensate
+	for long term drift. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
+k_d
+	The derivative term of the power allocator governor's PID
+	controller. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
+integral_cutoff
+	Temperature offset from the desired temperature trip point
+	above which the integral term of the power allocator
+	governor's PID controller starts accumulating errors. For
+	example, if integral_cutoff is 0, then the integral term only
+	accumulates error when temperature is above the desired
+	temperature trip point. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
 *****************************
 * Cooling device attributes *
 *****************************
drivers/thermal/thermal_core.c +101 -0

113 lines not on this page · raw message

Signature · 1 line
1.9.1
#5 lore raw

PATCH v3 4/5 Revert "cpufreq: remove CPUFREQ_UPDATE_POLICY_CPU notifications"

In reply to Javi Merino

This reverts commit d9f354460db8 ("cpufreq: remove CPUFREQ_UPDATE_POLICY_CPU notifications"). When the lead cpu of a group of cpus managed by cpufreq is hotplugged out, the cpu device cached in drivers/thermal/cpu_cooling.c needs to be updated accordingly.

Bring back the CPUFREQ_UPDATE_POLICY_CPU notifier so that we can react to it.

Cc Viresh Kumar <[email protected]>
Cc Rafael J. Wysocki <[email protected]>

---
Hi Viresh,

We just noticed that you removed this from v4.0-rc1 when we were
planning on using it, that's why are reverting the patch. The user is
introduced in the next patch: "[PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu".

 drivers/cpufreq/cpufreq.c | 3 +++
 include/linux/cpufreq.h   | 5 +++--
 2 files changed, 6 insertions(+), 2 deletions(-)
drivers/cpufreq/cpufreq.c +3 -0
index 28e59a48b35f..215147050c3d 100644
@@ -1095,6 +1095,9 @@ static int update_policy_cpu(struct cpufreq_policy *policy, unsigned int cpu,
 	policy->cpu = cpu;
 	up_write(&policy->rwsem);
 
+	blocking_notifier_call_chain(&cpufreq_policy_notifier_list,
+			CPUFREQ_UPDATE_POLICY_CPU, policy);
+
 	return 0;
 }
 
include/linux/cpufreq.h +3 -2
index 2ee4888c1f47..7e1a389b4e92 100644
@@ -368,8 +368,9 @@ static inline void cpufreq_resume(void) {}
 #define CPUFREQ_INCOMPATIBLE		(1)
 #define CPUFREQ_NOTIFY			(2)
 #define CPUFREQ_START			(3)
-#define CPUFREQ_CREATE_POLICY		(4)
-#define CPUFREQ_REMOVE_POLICY		(5)
+#define CPUFREQ_UPDATE_POLICY_CPU	(4)
+#define CPUFREQ_CREATE_POLICY		(5)
+#define CPUFREQ_REMOVE_POLICY		(6)
 
 #ifdef CONFIG_CPU_FREQ
 int cpufreq_register_notifier(struct notifier_block *nb, unsigned int list);
Signature · 1 line
1.9.1
#6 lore raw

PATCH v3 5/5 thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

In reply to Javi Merino

From: Kapileshwar Singh <[email protected]>

When cpufreq changes the policy cpu, we need to update our cached cpu device accordingly.

Cc Zhang Rui <[email protected]>
Cc Eduardo Valentin <[email protected]>
---
 drivers/thermal/cpu_cooling.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/thermal/cpu_cooling.c +3 -0
index c4974144c787..e306d6bc3cf1 100644
@@ -269,6 +269,9 @@ static int cpufreq_thermal_notifier(struct notifier_block *nb,
 		mutex_unlock(&cooling_cpufreq_lock);
 		break;
 
+	case CPUFREQ_UPDATE_POLICY_CPU:
+		update_cpu_device(policy->cpu);
+		break;
 	case CPUFREQ_CREATE_POLICY:
 		update_cpu_device(policy->cpu);
 		break;
Signature · 1 line
1.9.1
maintainer
#7 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Quoted 17 lines · Javi Merino

From: Kapileshwar Singh <[email protected]>

When cpufreq changes the policy cpu, we need to update our cached cpu device accordingly.

Cc Zhang Rui <[email protected]>
Cc Eduardo Valentin <[email protected]>
---
 drivers/thermal/cpu_cooling.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/thermal/cpu_cooling.c +0 -0
index c4974144c787..e306d6bc3cf1 100644

drivers/thermal/cpu_cooling.c static int cpufreq_thermal_notifier(struct notifier_block *nb, · line 277

Quoted 9 lines
drivers/thermal/cpu_cooling.c +3 -0
@@ -269,6 +269,9 @@ static int cpufreq_thermal_notifier(struct notifier_block *nb,
                mutex_unlock(&cooling_cpufreq_lock);
                break;
 
+       case CPUFREQ_UPDATE_POLICY_CPU:
+               update_cpu_device(policy->cpu);
+               break;
        case CPUFREQ_CREATE_POLICY:
                update_cpu_device(policy->cpu);
                break;

First of all, I wasn't able to find 3/5 on LKML and I looked at 3/7 from an earlier
version to look at how update_cpu_device() looks like.

What I couldn't understand is why do you need to update things if policy->cpu is changing ?

I am expecting a detailed answer here according to your design, and we may be able to work out without such updates. Lets see..

#8 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Hi Viresh!

Quoted 32 lines · Viresh Kumar
Quoted 17 lines · Javi Merino

From: Kapileshwar Singh <[email protected]>

When cpufreq changes the policy cpu, we need to update our cached cpu device accordingly.

Cc Zhang Rui <[email protected]>
Cc Eduardo Valentin <[email protected]>
---
 drivers/thermal/cpu_cooling.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/thermal/cpu_cooling.c +0 -0
index c4974144c787..e306d6bc3cf1 100644

drivers/thermal/cpu_cooling.c static int cpufreq_thermal_notifier(struct notifier_block *nb, · line 277

Quoted 9 lines
drivers/thermal/cpu_cooling.c +3 -0
@@ -269,6 +269,9 @@ static int cpufreq_thermal_notifier(struct notifier_block *nb,
                mutex_unlock(&cooling_cpufreq_lock);
                break;
 
+       case CPUFREQ_UPDATE_POLICY_CPU:
+               update_cpu_device(policy->cpu);
+               break;
        case CPUFREQ_CREATE_POLICY:
                update_cpu_device(policy->cpu);
                break;

First of all, I wasn't able to find 3/5 on LKML and I looked at 3/7 from an earlier
version to look at how update_cpu_device() looks like.

Quoted 3 lines

What I couldn't understand is why do you need to update things if policy->cpu is changing ?

We store the device pointer of the lead CPU (policy CPU) in a cpufreq domain as a part of the cpufreq_cooling_device data structure. There is one cpufreq_cooling_device per cpufreq domain.

We need the device to find out the current OPP for the cpufreq_cooling_device for our static power calculation.

        opp = opp_find_freq_exact(cpu_dev, freq_hz, true);
        voltage = dev_pm_opp_get_voltage(opp);

The problem we are trying to solve here is:

When this lead CPU gets hotplugged out, the device pointer becomes stale and the policy cpu for the cpufreq domain changes. We then store the new policy CPU's device pointer for the in cpufreq_cooling_device on the reception of a notification from cpufreq. Being open to your suggestions for any other possible ways to solve the problem..

Regards,
KP

Quoted 3 lines

I am expecting a detailed answer here according to your design, and we may be able to work out without such updates. Lets see..

maintainer
#9 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Quoted 10 lines · Kapileshwar Singh

We store the device pointer of the lead CPU (policy CPU) in a cpufreq domain as a part of the cpufreq_cooling_device data structure. There is one cpufreq_cooling_device per cpufreq domain.

We need the device to find out the current OPP for the cpufreq_cooling_device for our static power calculation.

        opp = opp_find_freq_exact(cpu_dev, freq_hz, true);
        voltage = dev_pm_opp_get_voltage(opp);
Quoted 6 lines

The problem we are trying to solve here is:

When this lead CPU gets hotplugged out, the device pointer becomes stale and the policy cpu for the cpufreq domain changes. We then store the new policy CPU's device pointer for the in cpufreq_cooling_device on the reception of a notification from cpufreq. Being open to your suggestions for any other possible ways to solve the problem..

I would have loved that if life was that simple :)

So, the OPP library today isn't that perfect and so is this doing rounds [1]. The problem is the OPPs are initialized per device today and even if they are shared by multiple CPUs, OPP library doesn't know about it.

So, if the policy->cpu goes away, OPP APIs on the new CPU will not work as OPPs are only initialized for one CPU and not for others within the same policy :)

The way cpufreq-dt is taking care of this is by saving cpu_dev of the first CPU for which OPPs are initialized and always using that even if the CPU goes away. And you need to do exactly that.

And please, do test such scenario before sending the patches again. As it would have simply failed in this case, have you given it a try ..

Once my patchset [1] is applied, life would be very simple and we can call OPP library for any CPU, but that is going to take some time.

--
viresh

[1] https://www.marc.info/?l=linaro-kernel&m=142364262800650&w=3

#10 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Quoted 28 lines · Viresh Kumar
Quoted 10 lines · Kapileshwar Singh

We store the device pointer of the lead CPU (policy CPU) in a cpufreq domain as a part of the cpufreq_cooling_device data structure. There is one cpufreq_cooling_device per cpufreq domain.

We need the device to find out the current OPP for the cpufreq_cooling_device for our static power calculation.

        opp = opp_find_freq_exact(cpu_dev, freq_hz, true);
        voltage = dev_pm_opp_get_voltage(opp);
Quoted 6 lines

The problem we are trying to solve here is:

When this lead CPU gets hotplugged out, the device pointer becomes stale and the policy cpu for the cpufreq domain changes. We then store the new policy CPU's device pointer for the in cpufreq_cooling_device on the reception of a notification from cpufreq. Being open to your suggestions for any other possible ways to solve the problem..

I would have loved that if life was that simple :)

So, the OPP library today isn't that perfect and so is this doing rounds [1]. The problem is the OPPs are initialized per device today and even if they are shared by multiple CPUs, OPP library doesn't know about it.

So, if the policy->cpu goes away, OPP APIs on the new CPU will not work as OPPs are only initialized for one CPU and not for others within the same policy :)

Quoted 6 lines

The way cpufreq-dt is taking care of this is by saving cpu_dev of the first CPU for which OPPs are initialized and always using that even if the CPU goes away. And you need to do exactly that.

And please, do test such scenario before sending the patches again. As it would have simply failed in this case, have you given it a try ..

Yes I indeed tested the case where we cache the device pointer of the CPU for which the OPP's are populated. When this CPU is hotplugged out, it invalidates the device pointer itself. Here are the error we get in dmesg:

..

<3>[67203.216774] opp_get_voltage: Invalid parameters
<3>[67203.326774] opp_get_voltage: Invalid parameters
<3>[67203.326774] opp_get_voltage: Invalid parameters

..

Which happens because:

unsigned long dev_pm_opp_get_voltage(struct dev_pm_opp *opp) {
..

        tmp_opp = rcu_dereference(opp);
        if (unlikely(IS_ERR_OR_NULL(tmp_opp)) || !tmp_opp->available)
                pr_err("%s: Invalid parameters\n", __func__);
        else

..

Which happens when

        opp = dev_pm_opp_find_freq_exact(cpufreq_device->cpu_dev, freq_hz,
                                         true);

returns a an erroneous or NULL OPP or the opp is unavailable (in the above condition)

Regards,
KP

Quoted 9 lines

Once my patchset [1] is applied, life would be very simple and we can call OPP library for any CPU, but that is going to take some time.

--
viresh

[1] https://www.marc.info/?l=linaro-kernel&m=142364262800650&w=3

maintainer
#11 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Quoted 2 lines · Kapileshwar Singh

Yes I indeed tested the case where we cache the device pointer of the CPU for which the OPP's are populated. When this CPU is hotplugged out, it invalidates the device pointer itself. Here are the error we get in dmesg:

What do you mean by 'invalidates the device pointer' ? that cpu_dev is NULL ?

Quoted 3 lines
<3>[67203.216774] opp_get_voltage: Invalid parameters
<3>[67203.326774] opp_get_voltage: Invalid parameters
<3>[67203.326774] opp_get_voltage: Invalid parameters

Have you handwritten them ? Why don't they precede with dev_pm_* ??

Quoted 9 lines

Which happens because:

unsigned long dev_pm_opp_get_voltage(struct dev_pm_opp *opp) {
..

        tmp_opp = rcu_dereference(opp);
        if (unlikely(IS_ERR_OR_NULL(tmp_opp)) || !tmp_opp->available)
                pr_err("%s: Invalid parameters\n", __func__);

This %s should print routine name ..

Quoted 9 lines

else
..

Which happens when

        opp = dev_pm_opp_find_freq_exact(cpufreq_device->cpu_dev, freq_hz,
                                         true);

returns a an erroneous or NULL OPP or the opp is unavailable (in the above condition)

Please goto the depth of this thing, as I don't think it should happen.

Over that I was asking you if you have tested the solution Javi gave, because OPPs
wouldn't have been initialized for other CPUs once policy->cpu goes down.

#12 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Quoted 5 lines · Viresh Kumar
Quoted 2 lines · Kapileshwar Singh

Yes I indeed tested the case where we cache the device pointer of the CPU for which the OPP's are populated. When this CPU is hotplugged out, it invalidates the device pointer itself. Here are the error we get in dmesg:

What do you mean by 'invalidates the device pointer' ? that cpu_dev is NULL ?

The cpu_dev is not NULL but we get an erroneous OPP back. We found the problem lies in the way we calculate the frequency for the cluster.

Quoted 5 lines
Quoted 3 lines
<3>[67203.216774] opp_get_voltage: Invalid parameters
<3>[67203.326774] opp_get_voltage: Invalid parameters
<3>[67203.326774] opp_get_voltage: Invalid parameters

Have you handwritten them ? Why don't they precede with dev_pm_* ??

I have not handwritten them, It was from a Linaro 3.10 based kernel when I first noticed this issue but the same problem exists in mainline.

Apologies for this I sent you an older trace which I had saved when I found the bug. Here is the trace I get from mainline

[ 5680.135339] dev_pm_opp_get_voltage: Invalid parameters
[ 5680.245528] dev_pm_opp_get_voltage: Invalid parameters
[ 5680.355432] dev_pm_opp_get_voltage: Invalid parameters
[ 5680.465521] dev_pm_opp_get_voltage: Invalid parameters
[ 5680.575599] dev_pm_opp_get_voltage: Invalid parameters
[ 5680.685817] dev_pm_opp_get_voltage: Invalid parameters
[ 5680.795556] dev_pm_opp_get_voltage: Invalid parameters
[ 5680.905598] dev_pm_opp_get_voltage: Invalid parameters
Quoted 4 lines
Quoted 4 lines

Which happens because:

Quoted 19 lines
Quoted 6 lines

unsigned long dev_pm_opp_get_voltage(struct dev_pm_opp *opp) {
..

        tmp_opp = rcu_dereference(opp);
        if (unlikely(IS_ERR_OR_NULL(tmp_opp)) || !tmp_opp->available)
                pr_err("%s: Invalid parameters\n", __func__);

This %s should print routine name ..

Quoted 9 lines

else
..

Which happens when

        opp = dev_pm_opp_find_freq_exact(cpufreq_device->cpu_dev, freq_hz,
                                         true);

returns a an erroneous or NULL OPP or the opp is unavailable (in the above condition)

Update: This returns an erroneous OPP

Quoted 5 lines

Please goto the depth of this thing, as I don't think it should happen.

Over that I was asking you if you have tested the solution Javi gave, because OPPs
wouldn't have been initialized for other CPUs once policy->cpu goes down.

I did test this but we were working with the assumption that OPPs should be populated for all the CPUs and also that OPPs are lost for a hotplugged CPU which I see is not the case.

We have looked at this more closely and found that problem lies in:

	freq = cpufreq_quick_get(cpumask_any(&cpufreq_device->allowed_cpus));

which returns a NULL frequency as we are not checking for online CPUs here. We shall come up with a fix for this. Many thanks for helping us with the investigation.

Regards,
KP

#13 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Quoted 1 line · Kapileshwar Singh

On 03/03/15 13:07, Viresh Kumar wrote:

[...]

Quoted 5 lines
Quoted 5 lines

Please goto the depth of this thing, as I don't think it should happen.

Over that I was asking you if you have tested the solution Javi gave, because OPPs
wouldn't have been initialized for other CPUs once policy->cpu goes down.

Quoted 8 lines

I did test this but we were working with the assumption that OPPs should be populated for all the CPUs and also that OPPs are lost for a hotplugged CPU which I see is not the case.

We have looked at this more closely and found that problem lies in:

        freq = cpufreq_quick_get(cpumask_any(&cpufreq_device->allowed_cpus));

which returns a NULL frequency as we are not checking for online CPUs here. We shall come up with a fix for this. Many thanks for helping us with the investigation.

You can use any_online_cpu(..) instead of cpumask_any IMO

Regards,
Sudeep

maintainer
#14 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Quoted 1 line · Sudeep Holla

You can use any_online_cpu(..) instead of cpumask_any IMO

What about policy->cpu ? :)

#15 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Quoted 5 lines · Viresh Kumar
Quoted 1 line · Sudeep Holla

You can use any_online_cpu(..) instead of cpumask_any IMO

What about policy->cpu ? :)

We can, was not sure if they have access to policy in their thermal driver. Just the first thought seeing one line of the code posted :)

Regards,
Sudeep

maintainer
#16 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

In reply to Kapileshwar Singh
Quoted 1 line · Kapileshwar Singh

I did test this but we were working with the assumption that OPPs should be populated for all the CPUs and also that OPPs are lost for a hotplugged CPU which I see is not the case.

Then what did you test? My point here is, even with the latest patches that you have
sent, you wouldn't be able to get the OPPs once policy->cpu goes down. So, how did this worked for you ?

Quoted 5 lines

We have looked at this more closely and found that problem lies in:

        freq = cpufreq_quick_get(cpumask_any(&cpufreq_device->allowed_cpus));

which returns a NULL frequency as we are not checking for online CPUs here. We shall come up with a fix for this. Many thanks for helping us with the investigation.

Right.

#17 lore raw

Re: [PATCH v3 5/5] thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu

Quoted 8 lines · Viresh Kumar
Quoted 1 line · Kapileshwar Singh

I did test this but we were working with the assumption that OPPs should be populated for all the CPUs and also that OPPs are lost for a hotplugged CPU which I see is not the case.

Then what did you test? My point here is, even with the latest patches that you have
sent, you wouldn't be able to get the OPPs once policy->cpu goes down. So, how did this worked for you ?

We were basing our fix on possibility of having OPPs for all the CPUs and we incorrectly attributed the erroneous OPP we got from dev_pm_opp_find_freq_exact to the missing OPPs in the other CPUs.

Quoted 9 lines
Quoted 6 lines

We have looked at this more closely and found that problem lies in:

        freq = cpufreq_quick_get(cpumask_any(&cpufreq_device->allowed_cpus));

which returns a NULL frequency as we are not checking for online CPUs here. We shall come up with a fix for this. Many thanks for helping us with the investigation.

Right.

#18 lore raw
In reply to Javi Merino
Quoted 10 lines · Javi Merino

*** BLURB HERE ***

Hi linux-pm,

Introduce the power allocator governor, a thermal governor that allocates device power to control temperature. This series is based on branch "linus" of Eduardo's linux-soc-thermal tree:

git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal.git

Quoted 8 lines

Changes since v2:
- Address Eduardo's review
+ Turn variable-size array in divvy_up_power() into a devm_kcalloc() as suggested by Eduardo
+ Remove #ifdeffery from thermal_core.c as suggested by Eduardo
- Bring back cpufreq's CPUFREQ_UPDATE_POLICY_CPU notifier in order to update the cpu device in cpu_cooling.c when cpufreq changes the policy cpu.

Can you please elaborate a bit more on the issue you saw to decide to bring this functionality back?

Can we keep the cpufreq changes as a separated thread? To me if the issue happens to power allocator, it also happens to the other governors. I don't want to block power allocator due to the cpufreq chanes.

Besides, cpufreq changes should go via the proper cpufreq tree, not the thermal tree.

Quoted 97 lines

Changes since v1:
- Removed a memory leak pointed out by Steve Rostedt
- Relax the constraint of the trip points to be the first and last passive trip point as Lina Iyer suggested.
- Update how the governor treats weights to mimic the behavior of fair share in v3 of the weight fixes series.
- Don't cache the cpufreq_cooling_device and scan for it whenever we need it.
- Consider the rate of changes in the derivative term of the PID controller

Changes since RFC v6:
- Addressed Eduardo's review
+ Pass the interval to the static power function as suggested by Eduardo
+ Make the cooling device ops return 0 or -E* and put the calculation in a parameter, like the rest of the cooling device ops
+ Documentation improvements
- Use thermal_cdev_update() to change cooling device states
- Add a patch to export the power allocator governor's tzp parameters to sysfs

Changes since RFC v5:
- Addressed Stephen's review of the trace patches.
- Removed power actors and extended the cooling device interface instead.
- Let platforms override the power allocator governor parameters in their thermal zone parameters

Changes since RFC v4:
- Add more tracing
- Document some of the limitations of the power allocator governor
- Export the power_actor API and move power_actor.h to include/linux

Changes since RFC v3:
- Use tz->passive to poll faster when the first trip point is hit.
- Don't make a special directory for power_actors
- Add a DT property for sustainable-power
- Simplify the static power interface and pass the current thermal zone in every power_actor_ops to remove the controversial enum power_actor_types
- Use locks with the actor_list list
- Use cpufreq_get() to get the frequency of the cpu instead of using the notifiers.
- Remove the prompt for THERMAL_POWER_ACTOR_CPU when configuring the kernel

Changes since RFC v2:
- Changed the PI controller into a PID controller
- Added static power to the cpu power model
- tz parameter max_dissipatable_power renamed to sustainable_power
- Register the cpufreq cooling device as part of the power_cpu_actor registration.

Changes since RFC v1:
- Fixed finding cpufreq cooling devices in cpufreq_frequency_change()
- Replaced the cooling device interface with a separate power actor API
- Addressed most of Eduardo's comments
- Incorporated ftrace support for bitmask to trace cpumasks

Cheers,
Javi & Punit

Javi Merino (4):
thermal: introduce the Power Allocator governor
thermal: add trace events to the power allocator governor
thermal: export thermal_zone_parameters to sysfs Revert "cpufreq: remove CPUFREQ_UPDATE_POLICY_CPU notifications"

Kapileshwar Singh (1):
thermal: cpu_cooling: update the cpu device when cpufreq updates the
policy cpu

 Documentation/thermal/power_allocator.txt      | 247 ++++++++++++
 Documentation/thermal/sysfs-api.txt            |  52 +++
 drivers/cpufreq/cpufreq.c                      |   3 +
 drivers/thermal/Kconfig                        |  15 +
 drivers/thermal/Makefile                       |   1 +
 drivers/thermal/cpu_cooling.c                  |  34 +-
 drivers/thermal/power_allocator.c              | 538 +++++++++++++++++++++++++
 drivers/thermal/thermal_core.c                 | 110 ++++-
 drivers/thermal/thermal_core.h                 |   8 +
 include/linux/cpufreq.h                        |   5 +-
 include/linux/thermal.h                        |  37 +-
 include/trace/events/thermal.h                 |  58 +++
 include/trace/events/thermal_power_allocator.h |  87 ++++
 13 files changed, 1184 insertions(+), 11 deletions(-)
 create mode 100644 Documentation/thermal/power_allocator.txt
 create mode 100644 drivers/thermal/power_allocator.c
 create mode 100644 include/trace/events/thermal_power_allocator.h
Signature · 1 line
1.9.1
#19 lore raw
Quoted 21 lines · Eduardo Valentin
Quoted 10 lines · Javi Merino

*** BLURB HERE ***

Hi linux-pm,

Introduce the power allocator governor, a thermal governor that allocates device power to control temperature. This series is based on branch "linus" of Eduardo's linux-soc-thermal tree:

git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal.git

Quoted 8 lines

Changes since v2:
- Address Eduardo's review
+ Turn variable-size array in divvy_up_power() into a devm_kcalloc() as suggested by Eduardo
+ Remove #ifdeffery from thermal_core.c as suggested by Eduardo
- Bring back cpufreq's CPUFREQ_UPDATE_POLICY_CPU notifier in order to update the cpu device in cpu_cooling.c when cpufreq changes the policy cpu.

Quoted 2 lines

Can you please elaborate a bit more on the issue you saw to decide to bring this functionality back?

I explained it in patch 5 ("thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu"): When cpufreq changes the policy cpu, the cpufreq cooling device needs to update the cached cpu device accordingly.

Quoted 3 lines

Can we keep the cpufreq changes as a separated thread? To me if the issue happens to power allocator, it also happens to the other governors.

The cached cpu device was introduced in patch 98ea0816118f ("thermal:
cpu_cooling: implement the power cooling device API") in your tree. I'm posting it here because it only affects doesn't affect other governors.

Quoted 2 lines

I don't want to block power allocator due to the cpufreq changes.

You don't have to. It's the last two patches precisely for that. You can merge everything up to patch 3 without depending on cpufreq changes.

Quoted 2 lines

Besides, cpufreq changes should go via the proper cpufreq tree, not the thermal tree.

It's a change to cpu_cooling (patch 5) that needs a revert of cpufreq (patch 4). The change to cpu_cooling depends on the patches already in your branch. So you will have to carry this patch in your branch with an Ack from the cpufreq maintainers.

Cheers,
Javi

#20 lore raw
Quoted 46 lines · Javi Merino
Quoted 21 lines · Eduardo Valentin
Quoted 10 lines · Javi Merino

*** BLURB HERE ***

Hi linux-pm,

Introduce the power allocator governor, a thermal governor that allocates device power to control temperature. This series is based on branch "linus" of Eduardo's linux-soc-thermal tree:

git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal.git

Quoted 8 lines

Changes since v2:
- Address Eduardo's review
+ Turn variable-size array in divvy_up_power() into a devm_kcalloc() as suggested by Eduardo
+ Remove #ifdeffery from thermal_core.c as suggested by Eduardo
- Bring back cpufreq's CPUFREQ_UPDATE_POLICY_CPU notifier in order to update the cpu device in cpu_cooling.c when cpufreq changes the policy cpu.

Quoted 2 lines

Can you please elaborate a bit more on the issue you saw to decide to bring this functionality back?

I explained it in patch 5 ("thermal: cpu_cooling: update the cpu device when cpufreq updates the policy cpu"): When cpufreq changes the policy cpu, the cpufreq cooling device needs to update the cached cpu device accordingly.

Quoted 3 lines

Can we keep the cpufreq changes as a separated thread? To me if the issue happens to power allocator, it also happens to the other governors.

The cached cpu device was introduced in patch 98ea0816118f ("thermal:
cpu_cooling: implement the power cooling device API") in your tree. I'm posting it here because it only affects doesn't affect other governors.

Quoted 2 lines

I don't want to block power allocator due to the cpufreq changes.

You don't have to. It's the last two patches precisely for that. You can merge everything up to patch 3 without depending on cpufreq changes.

Quoted 7 lines
Quoted 2 lines

Besides, cpufreq changes should go via the proper cpufreq tree, not the thermal tree.

It's a change to cpu_cooling (patch 5) that needs a revert of cpufreq (patch 4). The change to cpu_cooling depends on the patches already in your branch. So you will have to carry this patch in your branch with an Ack from the cpufreq maintainers.

Yes, the patches in the thermal code depend on code in my tree. However, changes in cpufreq are not depending on any code in my tree. Unless there is a direct dependency, I would prefer to send them via the correct tree.

I simple don't feel comfortable sending stuff that are really part of what is described in MAINTAINERS. But if you get an ack from cpufreq, then of course, I can consider it.

Quoted 3 lines

Cheers,
Javi