RAPL (Running Average Power Limit) driver
PATCH v2 1/1 RAPL (Running Average Power Limit) driver
Changes since V2:
- use 'struct device' instead of raw kobject to represent RAPL domains
- changed eventfd control interface to use event string instead of passing file descriptors that cannot be authenticated in sysfs directory
- clean ups based on v1 reviews
- use kcalloc for arrays
- drop dependencies on X86
- misc cleanups
I don have a checkpatch error which I think it is not valid. I can fix it by avoiding the macro/adding more lines.
ERROR: Macros with complex values should be enclosed in parenthesis #735: FILE: platform/x86/intel_rapl.c:735:
+#define RO_PRIMITIVE(name) \
+ primitive_show_fn(name) \
+ static DEVICE_ATTR(name, S_IRUGO, show_##name, NULL)
If i put parenthesis around it to mute the error then it won't compile.
I attempted to come up with a simple version as Greg suggested to just expose cooling device but then I realized we could lose many key functionalities partially due to the correlations among RAPL control knobs. I am still working on this since it is not as simple as deleting the advanced interfaces, I would have to do some setup based on best guesses.
Jacob Pan (1):
Introduce Intel RAPL cooling device driver
Documentation/ABI/testing/sysfs-class-intel-rapl | 121 ++ drivers/platform/x86/Kconfig | 9 + drivers/platform/x86/Makefile | 1 + drivers/platform/x86/intel_rapl.c | 1285 ++++++++++++++++++++++ drivers/platform/x86/intel_rapl.h | 244 ++++ 5 files changed, 1660 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-class-intel-rapl create mode 100644 drivers/platform/x86/intel_rapl.c create mode 100644 drivers/platform/x86/intel_rapl.h
Signature · 1 line
1.7.9.5
PATCH v2 1/1 Introduce Intel RAPL cooling device driver
RAPL(Running Average Power Limit) interface provides platform software with the ability to monitor, control, and get notifications on SOC power consumptions. Since its first appearance on Sandy Bridge, more features have being added to extend its usage. In RAPL, platforms are divided into domains for fine grained control. These domains include package, DRAM controller, CPU core (Power Plane 0), graphics uncore (power plane 1), etc.
The purpose of this driver is to expose RAPL for userspace consumption. Overall, RAPL fits in the generic thermal layer in that platform level power capping and monitoring are mainly used for thermal management and thermal layer provides the abstracted interface needed to have portable applications.
Specifically, userspace is presented with per domain cooling device with sysfs links to its true device. Although RAPL domain provides many parameters for fine tuning, long term power limit is exposed as the single knob via cooling device state. Whereas the rest of the parameters are still accessible via the linked RAPL class devices. This interface allows both simple and advanced use cases.
Eventfd is used to provide notifications to the userspace. At per domain level, use can choose any event capable parameters to register for threshold crossing notifications. This is shamelessly "borrowed" from cgroup with some trimming/fitting.
Zhang, Rui's initial RAPL driver was used as a reference and starting point. Many thanks.
https://lkml.org/lkml/2011/5/26/93
Unlike the patch above, which is mainly for monitoring, this driver focus on the control and usability by user applications.
--- Documentation/ABI/testing/sysfs-class-intel-rapl | 121 ++ drivers/platform/x86/Kconfig | 9 + drivers/platform/x86/Makefile | 1 + drivers/platform/x86/intel_rapl.c | 1285 ++++++++++++++++++++++ drivers/platform/x86/intel_rapl.h | 244 ++++ 5 files changed, 1660 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-class-intel-rapl create mode 100644 drivers/platform/x86/intel_rapl.c create mode 100644 drivers/platform/x86/intel_rapl.h
Documentation/ABI/testing/sysfs-class-intel-rapl new file +121 -0
@@ -0,0 +1,121 @@ +What: /sys/class/rapl/polling_freq_hz +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: + Frequency in HZ used to poll RAPL date. Only activated when + user selects an event that need to be polled or power limit + is set for one of the RAPL domains. +Users: Any thermal/power management applications want to do power capping + and battery life management on modern Intel SOCs. + +What: /sys/class/rapl/time_unit_divisor +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Indicating time increment size + +What: /sys/class/rapl/energy_unit_divisor +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Indicating energy counter increment size + +What: /sys/class/rapl/power_unit_divisor +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Indicating power unit increment size, used in power limits + +What: /sys/class/rapl/xxx_domain/average_power +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Average power over the current sampling period in milliwatts + +What: /sys/class/rapl/xxx_domain/energy +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Total amount of energy consumed (in jules) since that last time + this counter is cleared. + +What: /sys/class/rapl/xxx_domain/event_control +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: For arm event thresholds on selected data. Write to this file + in the format of <eventfd> <command> <threshold value> + where command must be one of the event capable file names such + as average_power. e.g. if user creates and eventfd=3, the write: + "3 average_power 12000" + eventfd will be notified when average_power of this domain crosses + 12000mW on either directions. + +What: /sys/class/rapl/xxx_domain/max_power +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Maximum power (mwatt) derived from the electrical spec of + this domain + +What: /sys/class/rapl/xxx_domain/min_power +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Minimum power (mwatt) derived from the electrical spec of + this domain + +What: /sys/class/rapl/xxx_domain/lock +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Indicates whether power limit control is locked by FW. + 0:unlocked; 1:locked + +What: /sys/class/rapl/xxx_domain/pl1_enable +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Indicates whether long term power_limit_1 is enabled + 0:disabled; 1:enabled + +What: /sys/class/rapl/xxx_domain/pl2_enable +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Indicates whether short term power_limit_2 is enabled + 0:disabled; 1:enabled + +What: /sys/class/rapl/xxx_domain/thermal_spec_power +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Thermal specification power(mw) of the domain which is the + upper limit of what user can assign to power_limit1 + +What: /sys/class/rapl/xxx_domain/time_windowX +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: RAPL algorithm controlled time window(msec), similar to RC + circuit discharge time. + +What: /sys/class/rapl/xxx_domain/power_limitX +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Target power limit in mwatt + +What: /sys/class/rapl/xxx_domain/plX_clamp +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Whether allow going below OS-requested P/T state + 0:disabled; 1:enabled + +What: /sys/class/rapl/xxx_domain/throttle_time +Date: April 2013 +KernelVersion: 3.10 +Contact: Jacob Pan <[email protected]> +Description: Accumulated throttle time (msec) done by RAPL control
drivers/platform/x86/Kconfig +9 -0
@@ -781,4 +781,13 @@ config APPLE_GMUX graphics as well as the backlight. Currently only backlight control is supported by the driver. +config INTEL_RAPL + tristate "Intel RAPL Support" + depends on THERMAL + default y + ---help--- + RAPL (Running Average Power Limit) provides mechanisms to enforce + and monitor power limits of supported Intel CPUs. Limits can be + set by RAPL domains such as package, core, graphics, etc. + endif # X86_PLATFORM_DEVICES
drivers/platform/x86/Makefile +1 -0
@@ -51,3 +51,4 @@ obj-$(CONFIG_INTEL_OAKTRAIL) += intel_oaktrail.o obj-$(CONFIG_SAMSUNG_Q10) += samsung-q10.o obj-$(CONFIG_APPLE_GMUX) += apple-gmux.o obj-$(CONFIG_CHROMEOS_LAPTOP) += chromeos_laptop.o +obj-$(CONFIG_INTEL_RAPL) += intel_rapl.o
drivers/platform/x86/intel_rapl.c new file +1285 -0
1,285 lines not on this page · raw message
drivers/platform/x86/intel_rapl.h new file +244 -0
244 lines not on this page · raw message
Signature · 1 line
1.7.9.5
Re: [PATCH v2] Introduce Intel RAPL cooling device driver
Quoted 26 lines · Jacob Pan
RAPL(Running Average Power Limit) interface provides platform software with the ability to monitor, control, and get notifications on SOC power consumptions. Since its first appearance on Sandy Bridge, more features have being added to extend its usage. In RAPL, platforms are divided into domains for fine grained control. These domains include package, DRAM controller, CPU core (Power Plane 0), graphics uncore (power plane 1), etc.
The purpose of this driver is to expose RAPL for userspace consumption. Overall, RAPL fits in the generic thermal layer in that platform level power capping and monitoring are mainly used for thermal management and thermal layer provides the abstracted interface needed to have portable applications.
Specifically, userspace is presented with per domain cooling device with sysfs links to its true device. Although RAPL domain provides many parameters for fine tuning, long term power limit is exposed as the single knob via cooling device state. Whereas the rest of the parameters are still accessible via the linked RAPL class devices. This interface allows both simple and advanced use cases.
Eventfd is used to provide notifications to the userspace. At per domain level, use can choose any event capable parameters to register for threshold crossing notifications. This is shamelessly "borrowed" from cgroup with some trimming/fitting.
Quoted 8 lines
Zhang, Rui's initial RAPL driver was used as a reference and starting point. Many thanks.
https://lkml.org/lkml/2011/5/26/93Unlike the patch above, which is mainly for monitoring, this driver focus on the control and usability by user applications.
Signed-off-by Jacob Pan <[email protected]>
As described in the cover letter response:
please redo this based on those comments.
greg k-h
Re: [PATCH v2] Introduce Intel RAPL cooling device driver
In reply to Jacob PanQuoted 7 lines · Jacob Pan
RAPL(Running Average Power Limit) interface provides platform software with the ability to monitor, control, and get notifications on SOC power consumptions. Since its first appearance on Sandy Bridge, more features have being added to extend its usage. In RAPL, platforms are divided into domains for fine grained control. These domains include package, DRAM controller, CPU core (Power Plane 0), graphics uncore (power plane 1), etc.
Some more trivia...
drivers/platform/x86/intel_rapl.c
Quoted 1 line
drivers/platform/x86/intel_rapl.c +0
-0
[]
drivers/platform/x86/intel_rapl.c line 191
Quoted 2 lines
drivers/platform/x86/intel_rapl.c +2
-0+/* in the order of enum rapl_primitives */ +static struct rapl_primitive_info rpi[] = {
const?
drivers/platform/x86/intel_rapl.c
Quoted 7 lines
drivers/platform/x86/intel_rapl.c +6
-0+/* name, mask, shift, msr index, unit divisor*/ +PRIMITIVE_INFO_INIT (energy,ENERGY_STATUS_MASK ,0 , +RAPL_DOMAIN_MSR_STATUS ,ENERGY_UNIT , +RAPL_PRIMITIVE_EVENT_CAP ), +static int rapl_set_cur_state(struct thermal_cooling_device *cdev, +unsigned long state)
I think most of this would look nicer if you
adopted the net style of aligning multi-line
statements to the appropriate open parenthesis.
Quoted 6 lines
drivers/platform/x86/intel_rapl.c +6
-0+static ssize_t store_event_control (struct device *dev, +struct device_attribute *attr, +const char *buf, +size_t size) +{ +struct rapl_domain *rd =dev_get_drvdata (dev);
drivers/platform/x86/intel_rapl.c line 591
Quoted 10 lines
drivers/platform/x86/intel_rapl.c +10
-0+unsigned int efd, new_threshold; +struct file *efile =NULL ; +int ret =0 ; +int prim; +struct rapl_event *ep; +u64 val; +char cmd[MAX_PRIM_NAME ]; + +if (sscanf (buf,"%u %s %u" , &efd, cmd, &new_threshold) !=3 ) +return -EINVAL ;
This sscanf looks fragile.
buf = "1 some_really_long_name_longer_than_MAX_PRIM_NAME 2"
stack overrun.
Where does buf come from?
drivers/platform/x86/intel_rapl.c line 718
Quoted 3 lines
drivers/platform/x86/intel_rapl.c +3
-0+#define primitive_show_fn (n) \ + +#define primitive_store_fn(n) \
Can't both of these be consolidated into a 2 functions using offset_of and/or adding a string argument?
Quoted 1 line
+static struct attribute *all_attrs[] = {
const?
drivers/platform/x86/intel_rapl.c line 851
Quoted 13 lines
drivers/platform/x86/intel_rapl.c +11
-0+ &dev_attr_energy.attr, +static void rapl_update_domain_data(void) +{ +int i, j; +u64 val; +bool xlate; + +for (i =0 ; i < rg_data.nr_domains; i++) { +/* exclude non-raw primitives */ +for (j =0 ; j <NR_RAW_PRIMITIVES ; j++) + xlate = !!(rpi[j].unit);
You don't really need the !!. The compiler does that.
drivers/platform/x86/intel_rapl.c line 883
Quoted 2 lines
drivers/platform/x86/intel_rapl.c +2
-0+/* for global rapl data */ +static struct class_attribute rapl_class_attrs[] = {
const?
Quoted 1 line
drivers/platform/x86/intel_rapl.c +1
-0+ GLOBAL_CLASS_RO_ATTR(energy_unit_divisor),
Re: [PATCH v2] Introduce Intel RAPL cooling device driver
Quoted 20 lines · Joe Perches
Quoted 6 lines · Jacob Pan
drivers/platform/x86/intel_rapl.c new file +6
-0+static ssize_t store_event_control (struct device *dev, +struct device_attribute *attr, +const char *buf, +size_t size) +{ +struct rapl_domain *rd =dev_get_drvdata (dev);drivers/platform/x86/intel_rapl.c line 591
Quoted 10 lines
drivers/platform/x86/intel_rapl.c new file +10
-0+unsigned int efd, new_threshold; +struct file *efile =NULL ; +int ret =0 ; +int prim; +struct rapl_event *ep; +u64 val; +char cmd[MAX_PRIM_NAME ]; + +if (sscanf (buf,"%u %s %u" , &efd, cmd, &new_threshold) !=3 ) +return -EINVAL ;This sscanf looks fragile.
Quoted 5 lines
buf = "1
some_really_long_name_longer_than_MAX_PRIM_NAME2"stack overrun.
Where does buf come from?
It comes from the sysfs core, which limits it to a PAGE_SIZE. But yes, it does look fragile, and flat out wrong, but I'm not going into that just yet, as that whole api should just be deleted for now.
greg k-h
Re: [PATCH v2] Introduce Intel RAPL cooling device driver
Quoted 26 lines · On Fri, 5 Apr 2013 14:35:18 -0700 Greg Kroah-Hartman…
Quoted 25 lines · Joe Perches
Quoted 16 lines · Jacob Pan
drivers/platform/x86/intel_rapl.c new file +15
-0+static ssize_t store_event_control (struct device *dev, +struct device_attribute *attr, +const char *buf, +size_t size) +{ +struct rapl_domain *rd =dev_get_drvdata (dev); +unsigned int efd, new_threshold; +struct file *efile =NULL ; +int ret =0 ; +int prim; +struct rapl_event *ep; +u64 val; +char cmd[MAX_PRIM_NAME ]; + + if (sscanf(buf, "%u %s %u", &efd, cmd,
&new_threshold) != 3)Review
Quoted 1 line
(quoted diff) +1
-0+ return -EINVAL;This sscanf looks fragile.
buf = "1
some_really_long_name_longer_than_MAX_PRIM_NAME2"stack overrun.
Quoted 8 lines
Quoted 1 line
Where does buf come from?
It comes from the sysfs core, which limits it to a
PAGE_SIZE. But yes, it does look fragile, and flat out wrong, but I'm not going into that just yet, as that whole api should just be deleted for now.greg k-h
I am just curious to know besides fixing string length check, is the problem related to the fact that I have more than
one element for the event_control sysfs node?
i.e. <eventfd> <event_name> <threshold>
If so, the reason being I don't know if there is a standard way to a transaction via sysfs where I can commit multiple inputs atomically.
Signature · 3 lines
Thanks, Jacob
Re: [PATCH v2] Introduce Intel RAPL cooling device driver
Quoted 3 lines · Jacob Pan
If so, the reason being I don't know if there is a standard way to a transaction via sysfs where I can commit multiple inputs atomically.
That is by design.
Re: [PATCH v2] Introduce Intel RAPL cooling device driver
In reply to Joe PerchesQuoted 4 lines · On Fri, 05 Apr 2013 14:26:35 -0700 Joe Perches <[email protected]> wrote:
drivers/platform/x86/intel_rapl.c line 191
Quoted 2 lines
drivers/platform/x86/intel_rapl.c new file +2
-0+/* in the order of enum rapl_primitives */ +static struct rapl_primitive_info rpi[] = {const?
I do need to override one entry for a special case. The hardware uses a different bit location for the same lock functionality.
The other comments are well taken,
Signature · 3 lines
Thanks, Jacob
Quoted 10 lines · Jacob Pan
Changes since V2:
- use 'struct device' instead of raw kobject to represent RAPL domains
- changed eventfd control interface to use event string instead of passing file descriptors that cannot be authenticated in sysfs directory
- clean ups based on v1 reviews
- use kcalloc for arrays
- drop dependencies on X86
- misc cleanups
Please no, split this up into at least 2 patches, the first being the "standard" thermal driver, and the rest for the "custom" extensions you are proposing.
That way the first can get accepted easily, and into 3.10 (hopefully), so people can use it and not stall everything as we continue to iterate over the "custom" things.
As is, this driver isn't acceptable, sorry.
greg k-h