RAPL (Running Average Power Limit) driver
cover RAPL (Running Average Power Limit) driver
RAPL(Running Average Power Limit) interface provides platform software with the ability to monitor, control, and get notifications on SOC power consumptions. Since its first appearance on Sandy Bridge, more features have being added to extend its usage. In RAPL, platforms are divided into domains for fine grained control. These domains include package, DRAM controller, CPU core (Power Plane 0), graphics uncore (power plane 1), etc.
The purpose of this driver is to expose RAPL for userspace consumption. Overall, RAPL fits in the generic thermal layer in that platform level power capping and monitoring are mainly used for thermal management and thermal layer provides the abstracted interface needed to have portable applications.
Specifically, userspace is presented with per domain cooling device with sysfs links to its kobject. Although RAPL domain provides many parameters for fine tuning, long term power limit is exposed as the single knob via cooling device state. Whereas the rest of the parameters are still accessible via the linked kobject. This simplifies the interface for both simple and advanced use cases.
DETAILS
=======
1. sysfs layout
As an x86 platform driver, RAPL driver binds with supported CPU ids during probing phase. Once domains are discovered, kobjets are created for each domain which are also linked with cooling devices after its registration with the generic thermal layer.
e.g.package RAPL domain registered as cooling device #15, link "device" back to its kobject.
/sys/class/thermal/cooling_device15/
├── cur_state ├── device -> ../../../platform/intel_rapl/rapl_domains/package ├── max_state ├── power ├── subsystem -> ../../../../class/thermal ├── type └── uevent
In driver's private sysfs area, domains kobjects are grouped under a kset which exposes global data. /sys/devices/platform/intel_rapl/
├── driver -> ../../../bus/platform/drivers/intel_rapl ├── power ├── rapl_domains │ ├── package │ │ └── thermal_cooling -> ../../../../virtual/thermal/cooling_device15 │ ├── power_plane_0 │ │ └── thermal_cooling -> ../../../../virtual/thermal/cooling_device16 │ └── power_plane_1 │ └── thermal_cooling -> ../../../../virtual/thermal/cooling_device18 └── subsystem -> ../../../bus/platform
2. per domain parameters
These are the fine tuning parameters only used by advanced power/thermal management applications. Refer to Intel SDM ch14 for details.
root@chromoly:/sys/class/thermal/cooling_device15/device# grep . * domain_name:package energy:924228 lock:0 max_power:0 max_window:0 min_power:0 pl1_clamp:1 pl1_enable:1 pl2_clamp:0 pl2_enable:1 power:2276 power_limit1:12000 power_limit2:31250 thermal_spec_power:17000 throttle_time: time_window1:28000 time_window2:0
3. event notifications
RAPL driver uses eventfd to provide userspace notifications on selected events. A file node called "event_control" is created for each RAPL domain. User can write control file descriptor, eventfd descriptor, and threshold to event_control file. Then, user application can use poll/select or blocking read to get notifications from the driver. Multiple events are allowed for each domain but only a single threshold is accepted.
4. Usage Examples (assume the topology in the sysfs layout above)
- set power limit to package domain (whole SOC package) to 6w
root@chromoly:~# echo 6000 > /sys/class/thermal/cooling_device15/cur_state
- set power limit to pp1 domain (graphics) to 4w
root@chromoly:~# echo 4000 > /sys/class/thermal/cooling_device18/cur_state
- check the current power usage in mWatts of pp1 domain
root@chromoly:~# cat /sys/class/thermal/cooling_device18/cur_state 61
- set event notification when power consumption of graphics unit crosses 5w.
root@chromoly:~#
event_fd_listener /sys/class/thermal/cooling_device18/device/power 5000
(event_fd_listener opens control file power and creates an eventfd, then write efd, cfd, threshold to event_control file of the given domain)
Caveats:
1. Package power limit events are supported by legacy thermal reporting mechanism, which uses local APIC thermal vector to generate interrupts when targeted P-states are not honored by the HW/FW. This is tied to machine check reporting. Until RAPL is used, this notification is a rare exception. When RAPL power limit is set artifically low, this notification could result in unwanted interrupts for each power limit excursion. Therefore, RAPL driver attempts to turn off the power limit notification interrupt when user sets a power limit.
2. By Intel Software Developer's Manual, RAPL interface can report max/min power for certain domains. But in reality HW often reports 0 for max/min power. RAPL driver tackles this problem by using thermal specification power or current power limit1 when max power information is not available. The result is that the max_state of a RAPL cooling device can be based on thermal spec power or power limit 1.
3. Since RAPL is backed by FW. In case of FW failure or plain lack of support, setting RAPL power limit could result in silent failure. I don't have a good solution for that.
4. Data polling starts only when the following items are set
- power limit
- events
Jacob Pan (1):
Introduce Intel RAPL cooling device driver
drivers/platform/x86/Kconfig | 8 + drivers/platform/x86/Makefile | 1 + drivers/platform/x86/intel_rapl.c | 1323 +++++++++++++++++++++++++++++++++++++ drivers/platform/x86/intel_rapl.h | 249 +++++++ 4 files changed, 1581 insertions(+) create mode 100644 drivers/platform/x86/intel_rapl.c create mode 100644 drivers/platform/x86/intel_rapl.h
Signature · 1 line
1.7.9.5
PATCH 1/1 Introduce Intel RAPL cooling device driver
RAPL(Running Average Power Limit) interface provides platform software with the ability to monitor, control, and get notifications on SOC power consumptions. Since its first appearance on Sandy Bridge, more features have being added to extend its usage. In RAPL, platforms are divided into domains for fine grained control. These domains include package, DRAM controller, CPU core (Power Plane 0), graphics uncore (power plane 1), etc.
The purpose of this driver is to expose RAPL for userspace consumption. Overall, RAPL fits in the generic thermal layer in that platform level power capping and monitoring are mainly used for thermal management and thermal layer provides the abstracted interface needed to have portable applications.
Specifically, userspace is presented with per domain cooling device with sysfs links to its kobject. Although RAPL domain provides many parameters for fine tuning, long term power limit is exposed as the single knob via cooling device state. Whereas the rest of the parameters are still accessible via the linked kobject. This simplifies the interface for both simple and advanced use cases.
Eventfd is used to provide notifications to the userspace. At per domain level, use can choose any event capable parameters to register for threshold crossing notifications. This is shamelessly "borrowed" from cgroup with some trimming/fitting.
Zhang, Rui's initial RAPL driver was used as a reference and starting point. Many thanks.
https://lkml.org/lkml/2011/5/26/93
Unlike the patch above, which is mainly for monitoring, this driver focus on the control and usability by user applications.
--- drivers/platform/x86/Kconfig | 8 + drivers/platform/x86/Makefile | 1 + drivers/platform/x86/intel_rapl.c | 1323 +++++++++++++++++++++++++++++++++++++ drivers/platform/x86/intel_rapl.h | 249 +++++++ 4 files changed, 1581 insertions(+) create mode 100644 drivers/platform/x86/intel_rapl.c create mode 100644 drivers/platform/x86/intel_rapl.h
drivers/platform/x86/Kconfig +8 -0
@@ -781,4 +781,12 @@ config APPLE_GMUX graphics as well as the backlight. Currently only backlight control is supported by the driver. +config INTEL_RAPL + tristate "Intel RAPL Support" + depends on X86 && THERMAL + default y + ---help--- + RAPL, AKA, Running Average Power Limit provides mechanisms to enforce + and monitor per domain power consumption limits of supported Intel CPUs. + endif # X86_PLATFORM_DEVICES
drivers/platform/x86/Makefile +1 -0
@@ -51,3 +51,4 @@ obj-$(CONFIG_INTEL_OAKTRAIL) += intel_oaktrail.o obj-$(CONFIG_SAMSUNG_Q10) += samsung-q10.o obj-$(CONFIG_APPLE_GMUX) += apple-gmux.o obj-$(CONFIG_CHROMEOS_LAPTOP) += chromeos_laptop.o +obj-$(CONFIG_INTEL_RAPL) += intel_rapl.o
drivers/platform/x86/intel_rapl.c new file +1323 -0
1,323 lines not on this page · raw message
drivers/platform/x86/intel_rapl.h new file +249 -0
249 lines not on this page · raw message
Signature · 1 line
1.7.9.5
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 6 lines · Jacob Pan
drivers/platform/x86/intel_rapl.c new file +0
-0
drivers/platform/x86/intel_rapl.c line 48
Quoted 10 lines
drivers/platform/x86/intel_rapl.c new file +9
-0@@ -0,0 +1,1323 @@ +/* +* intel_rapl.c - Intel Running Average Power Limit Driver for MSR based +* RAPL interface +* +* Copyright (c) 2013, Intel Corporation. +* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +#include <linux/sysfs.h> + +#include "../../../fs/sysfs/sysfs.h"
What does this driver need from ^^^^^^ that file, which says:
* fs/sysfs/sysfs.h - sysfs internal header file
and should that be moved to include/linux/sysfs.h ?
Signature · 1 line
~Randy
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 16 lines · Randy Dunlap
Quoted 14 lines · Jacob Pan
drivers/platform/x86/intel_rapl.c new file +8
-0@@ -0,0 +1,1323 @@ +/* +* intel_rapl.c - Intel Running Average Power Limit Driver for MSR based +* RAPL interface +* +* Copyright (c) 2013, Intel Corporation. +* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +#include <linux/sysfs.h> +
Quoted 8 lines
drivers/platform/x86/intel_rapl.c line 48
Quoted 1 line
drivers/platform/x86/intel_rapl.c new file +1
-0+#include "../../../fs/sysfs/sysfs.h" What does this driver need from ^^^^^^ that file, which says:
*
fs/sysfs/sysfs.h- sysfs internal header fileand should that be moved to
include/linux/sysfs.h?
See my other complain about this already, I beat you to it :)
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
In reply to Jacob PanQuoted 7 lines · Jacob Pan
RAPL(Running Average Power Limit) interface provides platform software with the ability to monitor, control, and get notifications on SOC power consumptions. Since its first appearance on Sandy Bridge, more features have being added to extend its usage. In RAPL, platforms are divided into domains for fine grained control. These domains include package, DRAM controller, CPU core (Power Plane 0), graphics uncore (power plane 1), etc.
trivia only:
drivers/platform/x86/intel_rapl.c
Quoted 1 line
drivers/platform/x86/intel_rapl.c +0
-0
[]
drivers/platform/x86/intel_rapl.c line 77
Quoted 6 lines
drivers/platform/x86/intel_rapl.c +6
-0+static char *rapl_domain_names[] = { +"package" , +"power_plane_0" , +"power_plane_1" , +"dram" , +};
const
drivers/platform/x86/intel_rapl.c line 82
Quoted 3 lines
drivers/platform/x86/intel_rapl.c +3
-0+static void rapl_init_domains (void ) +{ +int i, j =0 ;
I think this below is unpleasant to read.
Maybe add a temporary pointer and get rid of j?
typeof rapl_domains *t = rapl_domains;
(not typeof, but the actual type)
drivers/platform/x86/intel_rapl.c line 90
Quoted 8 lines
drivers/platform/x86/intel_rapl.c +8
-0+ +for (i =0 ; i <RAPL_DOMAIN_MAX ; i++) { +unsigned int mask = rg_data.domain_map & (1 << i); + +switch (mask) { +case 1 <<RAPL_DOMAIN_PKG : + rapl_domains[j].name = + rapl_domain_names[RAPL_DOMAIN_PKG ];
t->name = rapl_domain_names[RAPL_DOMAIN_PKG]; etc...
[]
drivers/platform/x86/intel_rapl.c line 153
Quoted 2 lines
drivers/platform/x86/intel_rapl.c +2
-0+if (mask) + j++;
t++;
drivers/platform/x86/intel_rapl.c line 202
Quoted 2 lines
drivers/platform/x86/intel_rapl.c +2
-0+/* in the order of enum rapl_primitives */ +static struct rapl_primitive_info rpi[] = {
const?
drivers/platform/x86/intel_rapl.c line 206
Quoted 4 lines
drivers/platform/x86/intel_rapl.c +4
-0+/* name, mask, shift, msr index, unit divisor*/ +PRIMITIVE_INFO_INIT (energy,ENERGY_STATUS_MASK ,0 , +RAPL_DOMAIN_MSR_STATUS ,ENERGY_UNIT , + RAPL_PRIMITIVE_EVENT_CAP),
[]
drivers/platform/x86/intel_rapl.c line 260
Quoted 3 lines
drivers/platform/x86/intel_rapl.c +3
-0+static int rapl_read_data_raw (struct rapl_domain *domain, +struct rapl_primitive_info *rp,bool xlate,u64 *data) +{
[]
drivers/platform/x86/intel_rapl.c line 273
Quoted 1 line
drivers/platform/x86/intel_rapl.c +1
-0+/* specical-case pkg lock bit since pkg domain uses a different bit */
special-case
[]
Quoted 37 lines
drivers/platform/x86/intel_rapl.c +37
-0+static ssize_t store_event_control (struct rapl_domain *rd,const char *buf, +size_t size) +{ +unsigned int efd, cfd, new_threshold; +struct file *efile =NULL ; +struct file *cfile =NULL ; +int ret =0 ; +int prim; +struct rapl_event *ep; +u64 val; + +if (sscanf (buf,"%u %u %u" , &efd, &cfd, &new_threshold) !=3 ) +return -EINVAL ; + + efile =eventfd_fget (efd); +if (IS_ERR (efile)) { + ret =PTR_ERR (efile); +pr_err ("failed to get eventfd file %d\n" , efd); + goto done; + } + cfile =fget (cfd); +if (!cfile) { + ret = -EBADF ; +fput (efile); + goto done; + } +/* check if the cfile belongs to the same rapl domain */ +if (strcmp (rd->kobj.sd->s_name, + cfile->f_dentry->d_parent->d_name.name)) { +pr_debug ("cfile does not belong to domain %s\n" , + rd->kobj.sd->s_name); + ret = -EINVAL ; + goto exit_cleanup_fds; + } + prim =primitive_name_to_entry ( + (const char *)cfile->f_dentry->d_name.name); +if (prim <0 ) {
drivers/platform/x86/intel_rapl.c line 634
Quoted 10 lines
drivers/platform/x86/intel_rapl.c +10
-0+pr_err ("failed lookup primitive id for control file %s\n" , + cfile->f_dentry->d_name.name); + ret = -EINVAL ; + goto exit_cleanup_fds; + } +if (!(rpi[prim].flag &RAPL_PRIMITIVE_EVENT_CAP )) { +pr_info ("Invalid control file %d\n" , prim); + ret = -EINVAL ; + goto exit_cleanup_fds; + }
Perhaps all these pr_<levels> should be pr_err as they all lead to a failure path.
[]
drivers/platform/x86/intel_rapl.c line 1159
Quoted 3 lines
drivers/platform/x86/intel_rapl.c +3
-0+static int stop_periodic_polling (void ) +{ +if (true == polling_started) {
You don't need a comparison,
if (polling_started) {is perhaps better.
[]
drivers/platform/x86/intel_rapl.c line 1216
Quoted 2 lines
drivers/platform/x86/intel_rapl.c +2
-0+static int rapl_detect_domains (void ) +{
[]
drivers/platform/x86/intel_rapl.c line 1239
Quoted 1 line
drivers/platform/x86/intel_rapl.c +1
-0+pr_info ("Found %d vaild RAPL domains\n" , rg_data.nr_domains);
valid
drivers/platform/x86/intel_rapl.c
Quoted 2 lines
drivers/platform/x86/intel_rapl.c +2
-0+ rapl_domains =kzalloc (sizeof (struct rapl_domain ) * rg_data.nr_domains, +GFP_KERNEL );
kcalloc
drivers/platform/x86/intel_rapl.c line 1243
Quoted 2 lines
drivers/platform/x86/intel_rapl.c +2
-0+if (NULL == rapl_domains) { +pr_err ("Failed to allocate memory for rapl domain\n" );
You don't need OOM messages as the memory
subsystem already does a dump_stack on OOM.
[]
drivers/platform/x86/intel_rapl.c line 1267
Quoted 5 lines
drivers/platform/x86/intel_rapl.c +4
-0+static int __init intel_rapl_init (void ) +{ + rd_data =kzalloc (sizeof (struct rapl_domain_data ) * rg_data.nr_domains, +GFP_KERNEL );
kcalloc again.
drivers/platform/x86/intel_rapl.c line 1269
Quoted 2 lines
drivers/platform/x86/intel_rapl.c +2
-0+if (NULL == rd_data) { +pr_err ("Failed to allocate memory for rapl domain data\n" );
OOM too.
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
In reply to Jacob PanQuoted 48 lines · Jacob Pan
RAPL(Running Average Power Limit) interface provides platform software with the ability to monitor, control, and get notifications on SOC power consumptions. Since its first appearance on Sandy Bridge, more features have being added to extend its usage. In RAPL, platforms are divided into domains for fine grained control. These domains include package, DRAM controller, CPU core (Power Plane 0), graphics uncore (power plane 1), etc.
The purpose of this driver is to expose RAPL for userspace consumption. Overall, RAPL fits in the generic thermal layer in that platform level power capping and monitoring are mainly used for thermal management and thermal layer provides the abstracted interface needed to have portable applications.
Specifically, userspace is presented with per domain cooling device with sysfs links to its kobject. Although RAPL domain provides many parameters for fine tuning, long term power limit is exposed as the single knob via cooling device state. Whereas the rest of the parameters are still accessible via the linked kobject. This simplifies the interface for both simple and advanced use cases.
Eventfd is used to provide notifications to the userspace. At per domain level, use can choose any event capable parameters to register for threshold crossing notifications. This is shamelessly "borrowed" from cgroup with some trimming/fitting.
Zhang, Rui's initial RAPL driver was used as a reference and starting point. Many thanks.
https://lkml.org/lkml/2011/5/26/93Unlike the patch above, which is mainly for monitoring, this driver focus on the control and usability by user applications.
Signed-off-by Jacob Pan <[email protected]>--- drivers/platform/x86/Kconfig | 8 + drivers/platform/x86/Makefile | 1 + drivers/platform/x86/intel_rapl.c | 1323 +++++++++++++++++++++++++++++++++++++ drivers/platform/x86/intel_rapl.h | 249 +++++++ 4 files changed, 1581 insertions(+) create mode 100644 drivers/platform/x86/intel_rapl.c create mode 100644 drivers/platform/x86/intel_rapl.hdrivers/platform/x86/Kconfig +0
-0
drivers/platform/x86/Kconfig config APPLE_GMUX · line 789
Quoted 9 lines
drivers/platform/x86/Kconfig +6
-0@@ -781,4 +781,12 @@ config APPLE_GMUX graphics as well as the backlight. Currently only backlight control is supported by the driver. +config INTEL_RAPL + tristate "Intel RAPL Support" + depends on X86 && THERMAL + default y + ---help--- + RAPL, AKA, Running Average Power Limit provides mechanisms to enforce
RAPL, aka Running Average Power Limit, provides mechanisms to enforce
or a.k.a., but not AKA
or even:
RAPL (Running Average Power Limit) provides mechanisms to enforce
Quoted 3 lines
drivers/platform/x86/Kconfig +2
-0+ and monitor per domain power consumption limits of supported Intel CPUs. + endif # X86_PLATFORM_DEVICES
Signature · 1 line
~Randy
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
In reply to Jacob PanQuoted 34 lines · Jacob Pan
RAPL(Running Average Power Limit) interface provides platform software with the ability to monitor, control, and get notifications on SOC power consumptions. Since its first appearance on Sandy Bridge, more features have being added to extend its usage. In RAPL, platforms are divided into domains for fine grained control. These domains include package, DRAM controller, CPU core (Power Plane 0), graphics uncore (power plane 1), etc.
The purpose of this driver is to expose RAPL for userspace consumption. Overall, RAPL fits in the generic thermal layer in that platform level power capping and monitoring are mainly used for thermal management and thermal layer provides the abstracted interface needed to have portable applications.
Specifically, userspace is presented with per domain cooling device with sysfs links to its kobject. Although RAPL domain provides many parameters for fine tuning, long term power limit is exposed as the single knob via cooling device state. Whereas the rest of the parameters are still accessible via the linked kobject. This simplifies the interface for both simple and advanced use cases.
Eventfd is used to provide notifications to the userspace. At per domain level, use can choose any event capable parameters to register for threshold crossing notifications. This is shamelessly "borrowed" from cgroup with some trimming/fitting.
Zhang, Rui's initial RAPL driver was used as a reference and starting point. Many thanks.
https://lkml.org/lkml/2011/5/26/93Unlike the patch above, which is mainly for monitoring, this driver focus on the control and usability by user applications.
Signed-off-by Jacob Pan <[email protected]>
Quoted 8 lines
--- drivers/platform/x86/Kconfig | 8 + drivers/platform/x86/Makefile | 1 + drivers/platform/x86/intel_rapl.c | 1323 +++++++++++++++++++++++++++++++++++++ drivers/platform/x86/intel_rapl.h | 249 +++++++ 4 files changed, 1581 insertions(+) create mode 100644 drivers/platform/x86/intel_rapl.c create mode 100644 drivers/platform/x86/intel_rapl.h
You are creating new sysfs files, yet you forgot to document them in Documentation/ABI, please do so as part of this patch.
Quoted 13 lines
drivers/platform/x86/intel_rapl.c new file +10
-0@@ -0,0 +1,1323 @@ +/* +* intel_rapl.c - Intel Running Average Power Limit Driver for MSR based +* RAPL interface +* +* Copyright (c) 2013, Intel Corporation. +* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or (at
drivers/platform/x86/intel_rapl.c line 20
Quoted 10 lines
drivers/platform/x86/intel_rapl.c new file +10
-0+* your option) any later version. +* +* This program is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License along +* with this program; if not, write to the Free Software Foundation, Inc., +* 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
You really want to track the FSF's address for the next 40 years and keep your file up to date? I didn't think so...
drivers/platform/x86/intel_rapl.c line 48
Quoted 2 lines
drivers/platform/x86/intel_rapl.c new file +2
-0+#include "intel_rapl.h" +#include "../../../fs/sysfs/sysfs.h"
WTF?
Oh, that's a sure sign you are not doing something properly, if you think it's ok to muck around with the internals of sysfs.
There's a reason that file is "private", why do you think it's ok to use it directly? Did you just think that I somehow "forgot" to put it in the proper include directory?
Kids these days...
Surely one of the internal Intel code reviews caught this when you ran it through that process?
I'm not even going to review the rest of this file, please fix that up first, before sending it out again. And, when you do so, please cc: me.
Oh, I couldn't help myself:
drivers/platform/x86/intel_rapl.c line 909
Quoted 6 lines
drivers/platform/x86/intel_rapl.c new file +6
-0+static void rapl_domain_kobj_release (struct kobject *kobj) +{ +struct rapl_domain *rd =kobj_to_rapl_domain (kobj); + +complete (&rd->kobj_unregister); +}
No, that's not how you do a release function (hint, you can lock up your driver from userspace by holding your file open...) Free the memory here, where it is supposed to be free.
And, one final complaint, never use "raw" kobjects, for loads of good reasons, not the least being you just prevented userspace from seeing what is happening with your devices. Use the driver model, that's what it is there for, if you need "sub children", or subdirectories.
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 2 lines · Greg KH
I'm not even going to review the rest of this file, please fix that up first, before sending it out again. And, when you do so, please cc: me.
It's like a train wreck, you just can't tear away your eyes...
You race with userspace in the creation of your sysfs files. Your device is announced to userspace and then, later on, you create the sysfs files. This means that tools like libudev never sees your files, so the attributes aren't read properly by loads of userspace programs that you want to have read them.
Please fix that before resending this as well.
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
In reply to Greg KHQuoted 11 lines · On Tue, 2 Apr 2013 16:00:42 -0700 Greg KH <[email protected]> wrote:
drivers/platform/x86/intel_rapl.c line 48
Quoted 2 lines
drivers/platform/x86/intel_rapl.c new file +2
-0+#include "intel_rapl.h" +#include "../../../fs/sysfs/sysfs.h" WTF?
Oh, that's a sure sign you are not doing something properly, if you think it's ok to muck around with the internals of sysfs.
There's a reason that file is "private", why do you think it's ok to use it directly? Did you just think that I somehow "forgot" to put it in the proper include directory?
I did feel unsure about this but i saw some precedence in the kernel. Anyway, I needed a way to validate a userspace file passed to rapl driver belong to the same sysfs directory. I will look for alternative ways.
Signature · 3 lines
Thanks, Jacob
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 8 lines · Jacob Pan
Quoted 5 lines · On Tue, 2 Apr 2013 16:00:42 -0700 Greg KH <[email protected]> wrote:
drivers/platform/x86/intel_rapl.c line 48
Quoted 2 lines
drivers/platform/x86/intel_rapl.c new file +2
-0+#include "intel_rapl.h" +#include "../../../fs/sysfs/sysfs.h" WTF?
Quoted 7 lines
Quoted 6 lines
Oh, that's a sure sign you are not doing something properly, if you think it's ok to muck around with the internals of sysfs.
There's a reason that file is "private", why do you think it's ok to use it directly? Did you just think that I somehow "forgot" to put it in the proper include directory?
I did feel unsure about this but i saw some precedence in the kernel.
Someone else is doing this with the sysfs api? I don't see any other code in Linus's tree doing this at the moment, where did you see this? Let me know and I'll fix it up right away.
Quoted 3 lines
Anyway, I needed a way to validate a userspace file passed to rapl driver belong to the same sysfs directory. I will look for alternative ways.
What do you mean by this? What exactly are you trying to do? No normal driver code should _ever_ call sysfs functions directly, nor should they ever care about sysfs internals.
And, odds are, you didn't test your code as a module, right, as any internal sysfs function that you could get from this .h file, wouldn't be exported for a module to use, unless I missed one somewhere?
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 15 lines · On Tue, 2 Apr 2013 16:48:05 -0700 Greg KH <[email protected]> wrote:
Quoted 14 lines · Jacob Pan
Quoted 11 lines · On Tue, 2 Apr 2013 16:00:42 -0700 Greg KH <[email protected]> wrote:
drivers/platform/x86/intel_rapl.c line 48
Quoted 2 lines
drivers/platform/x86/intel_rapl.c new file +2
-0+#include "intel_rapl.h" +#include "../../../fs/sysfs/sysfs.h" WTF?
Oh, that's a sure sign you are not doing something properly, if you think it's ok to muck around with the internals of sysfs.
There's a reason that file is "private", why do you think it's ok to use it directly? Did you just think that I somehow "forgot" to put it in the proper include directory?
Quoted 7 lines
Quoted 2 lines
I did feel unsure about this but i saw some precedence in the kernel.
Someone else is doing this with the sysfs api? I don't see any other code in Linus's tree doing this at the moment, where did you see this? Let me know and I'll fix it up right away.
no, i did not mean sysfs api. I mean include internal header files via #include ../../
e.g.in drivers/usb/image/microtek.c
#include "../../scsi/scsi.h"
#include <scsi/scsi_host.h>
Quoted 8 lines
Quoted 3 lines
Anyway, I needed a way to validate a userspace file passed to rapl driver belong to the same sysfs directory. I will look for alternative ways.
What do you mean by this? What exactly are you trying to do? No normal driver code should _ever_ call sysfs functions directly, nor should they ever care about sysfs internals.
i did not call sysfs internal calls, just need to use struct sysfs_dirent {}
to do the following sanity check against user passed event control file, it is still not a 100% strong check.
/* check if the cfile belongs to the same rapl domain */
if (strcmp(rd->kobj.sd->s_name,
cfile->f_dentry->d_parent->d_name.name)) {
pr_debug("cfile does not belong to domain %s\n",
rd->kobj.sd->s_name);
ret = -EINVAL;
goto exit_cleanup_fds;
}Quoted 4 lines
And, odds are, you didn't test your code as a module, right, as any internal sysfs function that you could get from this .h file, wouldn't be exported for a module to use, unless I missed one somewhere?
I did run the driver as module since i didn't use sysfs internal functions, just the struct. I may be hitting a corner case here, but for drivers who need to discover sysfs hierarchy would it be useful to expose some info in struct sysfs_dirent{}?
Quoted 6 lines
greg k-h
--
To unsubscribe from this list: send the line "unsubscribe platform-driver-x86" in the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
[Jacob Pan]
Signature · 3 lines
Thanks, Jacob
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 25 lines · Jacob Pan
Quoted 15 lines · On Tue, 2 Apr 2013 16:48:05 -0700 Greg KH <[email protected]> wrote:
Quoted 14 lines · Jacob Pan
Quoted 11 lines · On Tue, 2 Apr 2013 16:00:42 -0700 Greg KH <[email protected]> wrote:
drivers/platform/x86/intel_rapl.c line 48
Quoted 2 lines
drivers/platform/x86/intel_rapl.c new file +2
-0+#include "intel_rapl.h" +#include "../../../fs/sysfs/sysfs.h" WTF?
Oh, that's a sure sign you are not doing something properly, if you think it's ok to muck around with the internals of sysfs.
There's a reason that file is "private", why do you think it's ok to use it directly? Did you just think that I somehow "forgot" to put it in the proper include directory?
Quoted 7 lines
Quoted 2 lines
I did feel unsure about this but i saw some precedence in the kernel.
Someone else is doing this with the sysfs api? I don't see any other code in Linus's tree doing this at the moment, where did you see this? Let me know and I'll fix it up right away.
Quoted 6 lines
no, i did not mean sysfs api. I mean include internal header files via #include ../../
e.g.indrivers/usb/image/microtek.c#include "
../../scsi/scsi.h"
#include <scsi/scsi_host.h>
That is because this is a scsi host driver. Your code is not part of sysfs itself.
Quoted 13 lines
Quoted 8 lines
Quoted 3 lines
Anyway, I needed a way to validate a userspace file passed to rapl driver belong to the same sysfs directory. I will look for alternative ways.
What do you mean by this? What exactly are you trying to do? No normal driver code should _ever_ call sysfs functions directly, nor should they ever care about sysfs internals.
i did not call sysfs internal calls, just need to use struct
sysfs_dirent{}to do the following sanity check against user passed event control file, it is still not a 100% strong check.
Quoted 8 lines
/* check if the cfile belongs to the same rapl domain */
if (strcmp(rd->kobj.sd->s_name,
cfile->f_dentry->d_parent->d_name.name)) {
pr_debug("cfile does not belong to domain %s\n",
rd->kobj.sd->s_name);
ret = -EINVAL;
goto exit_cleanup_fds;
}This made it through a code review at Intel? Seriously? Come on, there's just so much wrong here, I don't know where to begin.
Hint, if you find yourself caring about the internals of sysfs in a device driver, you are doing something so wrong it's not funny. Do you see _any_ other driver doing anything like this? What makes this driver so special that it can do unexpected, and totally different things with sysfs?
Quoted 8 lines
Quoted 4 lines
And, odds are, you didn't test your code as a module, right, as any internal sysfs function that you could get from this .h file, wouldn't be exported for a module to use, unless I missed one somewhere?
I did run the driver as module since i didn't use sysfs internal functions, just the struct. I may be hitting a corner case here, but for drivers who need to discover sysfs hierarchy would it be useful to expose some info in struct sysfs_dirent{}?
No, not at all, why would a driver ever care about that? Somehow we have gotten by for the past 10+ years without needing it, why is your driver so different than the thousands of other Linux drivers?
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 59 lines · On Wed, 3 Apr 2013 09:30:52 -0700 Greg KH <[email protected]> wrote:
Quoted 26 lines · Jacob Pan
Quoted 16 lines · On Tue, 2 Apr 2013 16:48:05 -0700 Greg KH <[email protected]> wrote:
Quoted 15 lines · Jacob Pan
Quoted 12 lines · On Tue, 2 Apr 2013 16:00:42 -0700 Greg KH <[email protected]> wrote:
drivers/platform/x86/intel_rapl.c line 48
Quoted 2 lines
drivers/platform/x86/intel_rapl.c new file +2
-0+#include "intel_rapl.h" +#include "../../../fs/sysfs/sysfs.h" WTF?
Oh, that's a sure sign you are not doing something properly, if you think it's ok to muck around with the internals of sysfs.
There's a reason that file is "private", why do you think it's ok to use it directly? Did you just think that I somehow "forgot" to put it in the proper include directory?
Quoted 7 lines
Quoted 2 lines
I did feel unsure about this but i saw some precedence in the kernel.
Someone else is doing this with the sysfs api? I don't see any other code in Linus's tree doing this at the moment, where did you see this? Let me know and I'll fix it up right away.
Quoted 6 lines
no, i did not mean sysfs api. I mean include internal header files via #include ../../
e.g.indrivers/usb/image/microtek.c#include "
../../scsi/scsi.h"
#include <scsi/scsi_host.h>That is because this is a scsi host driver. Your code is not part of sysfs itself.
Quoted 13 lines
Quoted 8 lines
Quoted 3 lines
Anyway, I needed a way to validate a userspace file passed to rapl driver belong to the same sysfs directory. I will look for alternative ways.
What do you mean by this? What exactly are you trying to do? No normal driver code should _ever_ call sysfs functions directly, nor should they ever care about sysfs internals.
i did not call sysfs internal calls, just need to use struct
sysfs_dirent{}to do the following sanity check against user passed event control file, it is still not a 100% strong check.
Quoted 8 lines
/* check if the cfile belongs to the same rapl domain */ if (strcmp(rd->kobj.sd->s_name, cfile->f_dentry->d_parent->d_name.name)) { pr_debug("cfile does not belong to domain %s\n", rd->kobj.sd->s_name); ret = -EINVAL; goto exit_cleanup_fds; }
Quoted 9 lines
This made it through a code review at Intel? Seriously? Come on, there's just so much wrong here, I don't know where to begin.
Hint, if you find yourself caring about the internals of sysfs in a device driver, you are doing something so wrong it's not funny. Do you see _any_ other driver doing anything like this? What makes this driver so special that it can do unexpected, and totally different things with sysfs?
I admit that my knowledge in this area are limited. I appreciate your help and straightforward comments.
Perhaps the reason is that not many drivers use eventfd and its way to arm event thresholds. The userspace passed an eventfd, a file descriptor, and a threshold value to the RAPL driver. In order to authenticate the control file descriptor is a valid, I need to check
1. the control file is capable of generating events, e.g. it can be an energy counter but not a constant power_limit1
2. the control file belongs to the same RAPL domain since the name are reused in all RAPL domains. e.g. all domains have energy counter and events are per domain.
I used the sysfs directory check for #2. It is just an extra check. But I think can drop that check and user pick events based on its name string. The fact that user writes to the per domain control file implies the domain specificness of the event.
In similar eventfd usage in cgroup, it has its own fs so i does the check based on file ops of the control files.
Quoted 10 lines
Quoted 9 lines
Quoted 5 lines
And, odds are, you didn't test your code as a module, right, as any internal sysfs function that you could get from this .h file, wouldn't be exported for a module to use, unless I missed one somewhere?
I did run the driver as module since i didn't use sysfs internal functions, just the struct. I may be hitting a corner case here, but for drivers who need to discover sysfs hierarchy would it be useful to expose some info in struct sysfs_dirent{}?
Quoted 5 lines
No, not at all, why would a driver ever care about that? Somehow we have gotten by for the past 10+ years without needing it, why is your driver so different than the thousands of other Linux drivers?
greg k-h
ditto, I will drop that extra check.
Signature · 3 lines
Thanks, Jacob
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 71 lines · Jacob Pan
Quoted 59 lines · On Wed, 3 Apr 2013 09:30:52 -0700 Greg KH <[email protected]> wrote:
Quoted 26 lines · Jacob Pan
Quoted 16 lines · On Tue, 2 Apr 2013 16:48:05 -0700 Greg KH <[email protected]> wrote:
Quoted 15 lines · Jacob Pan
Quoted 12 lines · On Tue, 2 Apr 2013 16:00:42 -0700 Greg KH <[email protected]> wrote:
Review
Quoted 2 lines
(quoted diff) +2
-0+#include "intel_rapl.h" +#include "../../../fs/sysfs/sysfs.h"WTF?
Oh, that's a sure sign you are not doing something properly, if you think it's ok to muck around with the internals of sysfs.
There's a reason that file is "private", why do you think it's ok to use it directly? Did you just think that I somehow "forgot" to put it in the proper include directory?
Quoted 7 lines
Quoted 2 lines
I did feel unsure about this but i saw some precedence in the kernel.
Someone else is doing this with the sysfs api? I don't see any other code in Linus's tree doing this at the moment, where did you see this? Let me know and I'll fix it up right away.
Quoted 6 lines
no, i did not mean sysfs api. I mean include internal header files via #include ../../
e.g.indrivers/usb/image/microtek.c#include "
../../scsi/scsi.h"
#include <scsi/scsi_host.h>That is because this is a scsi host driver. Your code is not part of sysfs itself.
Quoted 13 lines
Quoted 8 lines
Quoted 3 lines
Anyway, I needed a way to validate a userspace file passed to rapl driver belong to the same sysfs directory. I will look for alternative ways.
What do you mean by this? What exactly are you trying to do? No normal driver code should _ever_ call sysfs functions directly, nor should they ever care about sysfs internals.
i did not call sysfs internal calls, just need to use struct
sysfs_dirent{}to do the following sanity check against user passed event control file, it is still not a 100% strong check.
Quoted 8 lines
/* check if the cfile belongs to the same rapl domain */ if (strcmp(rd->kobj.sd->s_name, cfile->f_dentry->d_parent->d_name.name)) { pr_debug("cfile does not belong to domain %s\n", rd->kobj.sd->s_name); ret = -EINVAL; goto exit_cleanup_fds; }Quoted 9 lines
This made it through a code review at Intel? Seriously? Come on, there's just so much wrong here, I don't know where to begin.
Hint, if you find yourself caring about the internals of sysfs in a device driver, you are doing something so wrong it's not funny. Do you see _any_ other driver doing anything like this? What makes this driver so special that it can do unexpected, and totally different things with sysfs?
Quoted 5 lines
I admit that my knowledge in this area are limited. I appreciate your help and straightforward comments.
Perhaps the reason is that not many drivers use eventfd and its way to arm event thresholds.
sysfs should not be using eventfs, that's the issue here. Don't do that, sysfs has other apis to do this type of thing, if you somehow think it is something that you really need to do (hint, I doubt it...)
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
In reply to Greg KHQuoted 4 lines · On Tue, 2 Apr 2013 16:00:42 -0700 Greg KH <[email protected]> wrote:
And, one final complaint, never use "raw" kobjects, for loads of good reasons, not the least being you just prevented userspace from seeing what is happening with your devices. Use the driver model, that's what it is there for, if you need "sub children", or subdirectories.
I chose to use to kobjects for the reason that userspace can see the device linking more clearly.
Let me try to paraphrase, I have two options:
1. if I use the platform device model instead of raw kobjects, I would have one platform device for each rapl domain. Then link individual platform device with the generic thermal layer sysfs.
2. In the current patch, I have one platform driver, then expose per domain kobject that can be linked to the generic thermal layer. Common attributes of all domains are grouped under the kset.
I did consider both options. I thought using #2 option is better since it allow user to discover the topologies easier by following the sysfs link. If i use use #1, it would be hard to expose the common attributes and more code too. Perhpas I misread Documentation/kobjects.txt which i thought kobject/kset are perfect for presenting situation like this.
Signature · 3 lines
Thanks, Jacob
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 9 lines · Jacob Pan
Quoted 4 lines · On Tue, 2 Apr 2013 16:00:42 -0700 Greg KH <[email protected]> wrote:
And, one final complaint, never use "raw" kobjects, for loads of good reasons, not the least being you just prevented userspace from seeing what is happening with your devices. Use the driver model, that's what it is there for, if you need "sub children", or subdirectories.
I chose to use to kobjects for the reason that userspace can see the device linking more clearly.
Then use a 'struct device'.
Quoted 4 lines
Let me try to paraphrase, I have two options:
1. if I use the platform device model instead of raw kobjects, I would have one platform device for each rapl domain. Then link individual platform device with the generic thermal layer sysfs.
I don't understand.
Quoted 3 lines
2. In the current patch, I have one platform driver, then expose per domain kobject that can be linked to the generic thermal layer. Common attributes of all domains are grouped under the kset.
Then use a 'struct device' that is attached to that kset, no need to use a platform device, as that's not what is needed here, right?
Quoted 6 lines
I did consider both options. I thought using #2 option is better since it allow user to discover the topologies easier by following the sysfs link. If i use use #1, it would be hard to expose the common attributes and more code too. Perhpas I misread
Documentation/kobjects.txtwhich i thought kobject/kset are perfect for presenting situation like this.
No, kobjects are for things not using the device tree, like filesystems or other stuff. You are in the device tree, so use a 'struct device'. If you use a "raw" kobject, you do not notify userspace what is going on, and you loose out on a bunch of other stuff that the driver model provides you.
But do you even need to use anything at all?
Let's step back and start over, what exactly are you trying to tell userspace? What data do you have that you need to express to it? How do you want userspace to see/use it?
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 41 lines · On Tue, 2 Apr 2013 17:13:00 -0700 Greg KH <[email protected]> wrote:
Quoted 10 lines · Jacob Pan
Quoted 5 lines · On Tue, 2 Apr 2013 16:00:42 -0700 Greg KH <[email protected]> wrote:
And, one final complaint, never use "raw" kobjects, for loads of good reasons, not the least being you just prevented userspace from seeing what is happening with your devices. Use the driver model, that's what it is there for, if you need "sub children", or subdirectories.
I chose to use to kobjects for the reason that userspace can see the device linking more clearly.
Then use a 'struct device'.
Quoted 4 lines
Let me try to paraphrase, I have two options:
1. if I use the platform device model instead of raw kobjects, I would have one platform device for each rapl domain. Then link individual platform device with the generic thermal layer sysfs.I don't understand.
Quoted 3 lines
2. In the current patch, I have one platform driver, then expose per domain kobject that can be linked to the generic thermal layer. Common attributes of all domains are grouped under the kset.
Then use a 'struct device' that is attached to that kset, no need to use a platform device, as that's not what is needed here, right?
Quoted 6 lines
I did consider both options. I thought using #2 option is better since it allow user to discover the topologies easier by following the sysfs link. If i use use #1, it would be hard to expose the common attributes and more code too. Perhpas I misread
Documentation/kobjects.txtwhich i thought kobject/kset are perfect for presenting situation like this.No, kobjects are for things not using the device tree, like filesystems or other stuff. You are in the device tree, so use a 'struct device'. If you use a "raw" kobject, you do not notify userspace what is going on, and you loose out on a bunch of other stuff that the driver model provides you.
Quoted 7 lines
But do you even need to use anything at all?
Let's step back and start over, what exactly are you trying to tell userspace? What data do you have that you need to express to it? How do you want userspace to see/use it?
greg k-h
It is a good idea to step back and let me explain what I wanted to do here for userspace.
I have two kinds of applications that might use this driver.
1. simple use case where user sets a power limit for a RAPL domain. e.g. set graphics unit power limit to 7w
2. advanced use case where use can do fine tuning on top of simple power limit,e.g. the dynamic response parameters of power control logic, event notifications, etc.
For #1, this driver register with the abstract generic thermal layer (/sys/class/thermal) and presents itself as a set of cooling devices with a single knob per domain for power limits.
root@chromoly:/sys/class/thermal/cooling_device15# echo 7000 > cur_state
For #2, to give userspace complete control of the RAPL interface, which is not generic, I put them under the device private sysfs area.
root@chromoly:/sys/class/thermal/cooling_device15/device# echo 1000 > time_window1
As you mentioned about using device tree vs. fs, and how kobject are used for fs. I do have the need to go between a generic thermal sysfs and the true device tree. This is the reason why I used kobjects and link them between device tree and its thermal sysfs representation.
e.g. a RAPL package cooling device linked with its platform device kobj. (device is linked with rapl_domains/package, the line is too long)
root@chromoly:/sys/class/thermal# ls -l cooling_device15/ total 0 -rw-r--r-- 1 root root 4096 Apr 2 15:03 cur_state lrwxrwxrwx 1 root root 0 Apr 2 21:28 device -> ../../../platform/intel_rapl/rapl_domains/package -r--r--r-- 1 root root 4096 Apr 2 15:03 max_state drwxr-xr-x 2 root root 0 Apr 2 21:28 power lrwxrwxrwx 1 root root 0 Apr 2 15:03 subsystem -> ../../../../class/thermal -r--r--r-- 1 root root 4096 Apr 2 15:03 type -rw-r--r-- 1 root root 4096 Apr 2 15:03 uevent
For userspace which is not satisfied with the simple use case of a single knob for setting power limit, it can follow the link to find the device tree entry. Then get access to the complete knobs, including event notifications.
I wanted to use generic thermal fs in that it gives a simple representation of the most usable feature of RAPL. Basically, RAPL domains can be enumerated by thermal control apps just as ACPI fans or processor cooling.
Signature · 3 lines
Thanks, Jacob
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 14 lines · Jacob Pan
Quoted 3 lines
Let's step back and start over, what exactly are you trying to tell userspace? What data do you have that you need to express to it? How do you want userspace to see/use it?
It is a good idea to step back and let me explain what I wanted to do here for userspace.
I have two kinds of applications that might use this driver.
1. simple use case where user sets a power limit for a RAPL domain. e.g. set graphics unit power limit to 7w
2. advanced use case where use can do fine tuning on top of simple power limit,e.g. the dynamic response parameters of power control logic, event notifications, etc.
Quoted 4 lines
For #1, this driver register with the abstract generic thermal layer (
/sys/class/thermal) and presents itself as a set of cooling devices with a single knob per domain for power limits.root@chromoly:/sys/class/thermal/cooling_device15# echo 7000 > cur_state
Great, how about submitting that functionality as patch 1 of your series? That seems like a very "normal" thermal driver, right?
Quoted 3 lines
For #2, to give userspace complete control of the RAPL interface, which is not generic, I put them under the device private sysfs area.
root@chromoly:/sys/class/thermal/cooling_device15/device# echo 1000 > time_window1
I totally fail to understand the difference. What do you want to show to userspace that can't be expressed through the thermal interface today? Perhaps the thermal interface could be expanded to provide more functionality that you need? Why create a one-off API that will never be used again and require userspace programs to be written just to handle this one type of device?
Quoted 4 lines
As you mentioned about using device tree vs. fs, and how kobject are used for fs. I do have the need to go between a generic thermal sysfs and the true device tree. This is the reason why I used kobjects and link them between device tree and its thermal sysfs representation.
I don't understand your leap to using kobjects.
Quoted 12 lines
e.g. a RAPL package cooling device linked with its platform device kobj. (device is linked with rapl_domains/package, the line is too long)
root@chromoly:/sys/class/thermal# ls -l cooling_device15/ total 0 -rw-r--r-- 1 root root 4096 Apr 2 15:03 cur_state lrwxrwxrwx 1 root root 0 Apr 2 21:28 device -> ../../../platform/intel_rapl/rapl_domains/package -r--r--r-- 1 root root 4096 Apr 2 15:03 max_state drwxr-xr-x 2 root root 0 Apr 2 21:28 power lrwxrwxrwx 1 root root 0 Apr 2 15:03 subsystem -> ../../../../class/thermal
Review
Quoted 2 lines
(quoted diff) +0
-2-r--r--r-- 1 root root 4096 Apr 2 15:03 type -rw-r--r-- 1 root root 4096 Apr 2 15:03 uevent
I still don't understand. What are you adding here, the device symlink? Or something else?
Quoted 4 lines
For userspace which is not satisfied with the simple use case of a single knob for setting power limit, it can follow the link to find the device tree entry. Then get access to the complete knobs, including event notifications.
And what is in that device directory? What is rapl_domains? Why isn't that a normal 'struct device'?
Still confused.
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 15 lines · On Wed, 3 Apr 2013 09:35:09 -0700 Greg KH <[email protected]> wrote:
Quoted 14 lines · Jacob Pan
Quoted 3 lines
Let's step back and start over, what exactly are you trying to tell userspace? What data do you have that you need to express to it? How do you want userspace to see/use it?
It is a good idea to step back and let me explain what I wanted to do here for userspace.
I have two kinds of applications that might use this driver.
1. simple use case where user sets a power limit for a RAPL domain. e.g. set graphics unit power limit to 7w
2. advanced use case where use can do fine tuning on top of simple power limit,e.g. the dynamic response parameters of power control logic, event notifications, etc.
Quoted 9 lines
Quoted 5 lines
For #1, this driver register with the abstract generic thermal layer (
/sys/class/thermal) and presents itself as a set of cooling devices with a single knob per domain for power limits.root@chromoly:/sys/class/thermal/cooling_device15# echo 7000 > cur_stateGreat, how about submitting that functionality as patch 1 of your series? That seems like a very "normal" thermal driver, right?
yes, that would be a normal thermal cooling device driver. I will do that first. Thanks for the suggestion.
Quoted 8 lines
Quoted 4 lines
For #2, to give userspace complete control of the RAPL interface, which is not generic, I put them under the device private sysfs area. root@chromoly:/sys/class/thermal/cooling_device15/device# echo 1000 >
time_window1I totally fail to understand the difference. What do you want to show to userspace that can't be expressed through the thermal interface today?
The difference is one single knob (long term power limit) in the thermal interface vs multiple fine grained control and data in the complete RAPL interface.
Here is what a complete RAPL interface for package domain looks like.
root@chromoly:/sys/class/thermal/cooling_device15/device# grep . * domain_name:package energy:22396031 lock:0 max_power:0 max_window:0 min_power:0 pl1_clamp:0 pl1_enable:1 pl2_clamp:0 pl2_enable:1 power:7841 power_limit1:25000 power_limit2:31250 thermal_spec_power:17000 throttle_time: time_window1:28000 time_window2:0
Quoted 2 lines
Perhaps the thermal interface could be expanded to provide
more functionality that you need?
yes, some of them such as limits. But not all the data in the list above are suitable for thermal interface. That is why I am trying to balance between abstracted generic data and RAPL specific data while still allow linking between the two.
The way I envisioned how a thermal/power management app would use is:
1. go through generic thermal layer sysfs and find available RAPL domains
2. if the app wants to do more fine grained control, it follows the device symlink to locate the RAPL domain specific sysfs area.
Quoted 4 lines
Why create a one-off API that will
never be used again and require userspace programs to be written just to handle this one type of device?
why is that a one-off API? RAPL interface is maintained identical across Intel CPUs after Sandy Bridges. I agree with you that it is still one type of device with some of its data unique. Should i create a RAPL class device?
Quoted 8 lines
Quoted 5 lines
As you mentioned about using device tree vs. fs, and how kobject are used for fs. I do have the need to go between a generic thermal sysfs and the true device tree. This is the reason why I used kobjects and link them between device tree and its thermal sysfs representation.
I don't understand your leap to using kobjects.
I use kobjects mainly for its symlink to allow userspace locate the 'true' device behind generic thermal layers' cooling device.
Quoted 19 lines
Quoted 13 lines
e.g. a RAPL package cooling device linked with its platform device kobj. (device is linked with rapl_domains/package, the line is too long)
root@chromoly:/sys/class/thermal# ls -l cooling_device15/ total 0 -rw-r--r-- 1 root root 4096 Apr 2 15:03 cur_state lrwxrwxrwx 1 root root 0 Apr 2 21:28 device -> ../../../platform/intel_rapl/rapl_domains/package -r--r--r-- 1 root root 4096 Apr 2 15:03 max_state drwxr-xr-x 2 root root 0 Apr 2 21:28 power lrwxrwxrwx 1 root root 0 Apr 2 15:03 subsystem -> ../../../../class/thermalReview
Quoted 2 lines
(quoted diff) +0
-2-r--r--r-- 1 root root 4096 Apr 2 15:03 type -rw-r--r-- 1 root root 4096 Apr 2 15:03 ueventI still don't understand. What are you adding here, the device symlink? Or something else?
Quoted 6 lines
Quoted 4 lines
For userspace which is not satisfied with the simple use case of a single knob for setting power limit, it can follow the link to find the device tree entry. Then get access to the complete knobs, including event notifications.
And what is in that device directory?
the device directory contains the complete RAPL interface representation. paste the example of package domain again.
root@chromoly:/sys/class/thermal/cooling_device15/device# grep . * domain_name:package energy:22396031 lock:0 max_power:0 max_window:0 min_power:0 pl1_clamp:0 pl1_enable:1 pl2_clamp:0 pl2_enable:1 power:7841 power_limit1:25000 power_limit2:31250 thermal_spec_power:17000 grep: throttle_time: Input/output error time_window1:28000 time_window2:0
Quoted 3 lines
What is
rapl_domains? Why
isn't that a normal 'struct device'?
RAPL domains can be viewed as sub devices of RAPL interface. On a given platform they can be a complete or partial list of
package, power plane 0 (processor core), power plane 1 (graphics), dram controller, pch, etc.
Yes, I can create a RAPL class and create_device for each domain based on 'struct device'. I can still use the kobj symlink to link to generic thermal layer. I just thought it is an overkill as compared to simply using kobjects and kset.
I don't understand the benefit of using 'struct device' in this case. RAPL interface as a whole has its device, I am not creating standalone kobjects.
Quoted 3 lines
Still confused.
greg k-h
[Jacob Pan]
Signature · 3 lines
Thanks, Jacob
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 24 lines · Jacob Pan
Quoted 21 lines · On Wed, 3 Apr 2013 09:35:09 -0700 Greg KH <[email protected]> wrote:
Quoted 19 lines · Jacob Pan
Quoted 3 lines
Let's step back and start over, what exactly are you trying to tell userspace? What data do you have that you need to express to it? How do you want userspace to see/use it?
It is a good idea to step back and let me explain what I wanted to do here for userspace.
I have two kinds of applications that might use this driver.
1. simple use case where user sets a power limit for a RAPL domain. e.g. set graphics unit power limit to 7w
2. advanced use case where use can do fine tuning on top of simple power limit,e.g. the dynamic response parameters of power control logic, event notifications, etc.For #1, this driver register with the abstract generic thermal layer (
/sys/class/thermal) and presents itself as a set of cooling devices with a single knob per domain for power limits.root@chromoly:/sys/class/thermal/cooling_device15# echo 7000 > cur_state
Quoted 5 lines
Quoted 3 lines
Great, how about submitting that functionality as patch 1 of your series? That seems like a very "normal" thermal driver, right?
yes, that would be a normal thermal cooling device driver. I will do that first. Thanks for the suggestion.
Do that first, get it merged, and then let's work on the second part. The patch for that will be much more obvious as to what you are attempting to do.
Quoted 6 lines
Quoted 2 lines
Perhaps the thermal interface could be expanded to provide
more functionality that you need?yes, some of them such as limits. But not all the data in the list above are suitable for thermal interface. That is why I am trying to balance between abstracted generic data and RAPL specific data while still allow linking between the two.
What is not in the existing interface? And as this is a thermal device, why can't you add them there?
Quoted 5 lines
The way I envisioned how a thermal/power management app would use is:
1. go through generic thermal layer sysfs and find available RAPL domains
2. if the app wants to do more fine grained control, it follows the device symlink to locate the RAPL domain specific sysfs area.
So any application will have to know all of the device-specific attributes? That totally defeats the purpose of a generic api that the kernel is providing. You are creating device-specific apis that will not work over the long-run (i.e. next 5-10 years.) Please don't do that unless you have exhausted _all_ other alternatives.
So, get your first driver accepted, using the in-kernel thermal api, and then, if you still feel you wish to do device-specific extensions, we can discuss that then.
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 28 lines · On Fri, 5 Apr 2013 13:23:09 -0700 Greg KH <[email protected]> wrote:
Quoted 27 lines · Jacob Pan
Quoted 24 lines · On Wed, 3 Apr 2013 09:35:09 -0700 Greg KH <[email protected]> wrote:
Quoted 14 lines · Jacob Pan
Quoted 3 lines
Let's step back and start over, what exactly are you trying to tell userspace? What data do you have that you need to express to it? How do you want userspace to see/use it?
It is a good idea to step back and let me explain what I wanted to do here for userspace.
I have two kinds of applications that might use this driver.
1. simple use case where user sets a power limit for a RAPL domain. e.g. set graphics unit power limit to 7w
2. advanced use case where use can do fine tuning on top of simple power limit,e.g. the dynamic response parameters of power control logic, event notifications, etc.Quoted 5 lines
For #1, this driver register with the abstract generic thermal layer (
/sys/class/thermal) and presents itself as a set of cooling devices with a single knob per domain for power limits.root@chromoly:/sys/class/thermal/cooling_device15# echo 7000 > cur_stateGreat, how about submitting that functionality as patch 1 of your series? That seems like a very "normal" thermal driver, right?
Quoted 7 lines
Quoted 2 lines
yes, that would be a normal thermal cooling device driver. I will do that first. Thanks for the suggestion.
Do that first, get it merged, and then let's work on the second part. The patch for that will be much more obvious as to what you are attempting to do.
Sorry I was too busy to work on v2 before seeing this. I agree I need to simplify the interface, I just need to come up with a more intelligent way to abstract that and do the best guesses for the user. Hopefully, v2 will serve as a confirmation on the comments I got from v1. i.e. kobject->struct device, removed dependencies on sysfs internal data struct, etc.
Quoted 9 lines
Quoted 6 lines
Quoted 2 lines
Perhaps the thermal interface could be expanded to provide
more functionality that you need?yes, some of them such as limits. But not all the data in the list above are suitable for thermal interface. That is why I am trying to balance between abstracted generic data and RAPL specific data while still allow linking between the two.
What is not in the existing interface? And as this is a thermal device, why can't you add them there?
existing interface has only cur_state and max_state, I have been working with Rui (thermal maintainer) on adding more knobs for cooling devices, such as limit low, limit high, event control. I believe they can all be added.
What is not appropriate for thermal interface are the things like energy counters, accumulated throttle time, time constants that used by the internal control algorithm.
Quoted 7 lines
Quoted 6 lines
The way I envisioned how a thermal/power management app would use
is: 1. go through generic thermal layer sysfs and find available RAPL domains
2. if the app wants to do more fine grained control, it follows the device symlink to locate the RAPL domain specific sysfs area.
Quoted 6 lines
So any application will have to know all of the device-specific attributes? That totally defeats the purpose of a generic api that the kernel is providing. You are creating device-specific apis that will not work over the long-run (i.e. next 5-10 years.) Please don't do that unless you have exhausted _all_ other alternatives.
I agree we should improve the abstraction to avoid using device specific attributes. Unfortunately, on the other side power tuning tends to be very platform specific.
Quoted 4 lines
So, get your first driver accepted, using the in-kernel thermal api, and then, if you still feel you wish to do device-specific extensions, we can discuss that then.
Agreed. I just want to make sure the rapl class and per domain 'struct device' layout in v2 is what you suggested. I avoided using raw kobjects and sysfs internals.
The challenge for the simplified/abstract model is that the driver would have to guess what is best for the user.
e.g. when user selects power limit of 8200mw for the core power plane. Besides setting that limit, the driver will decide the following for the user:
- dynamics of the control, rise time, overshoot, steady state error
- whether or not allow P/T states to go below OS request value
- correlation with instantaneous power limit
I believe it is all doable, just need to strike a balance and for different platforms. But I do believe it is a better interface for most generic applications.
Quoted 6 lines
greg k-h
--
To unsubscribe from this list: send the line "unsubscribe platform-driver-x86" in the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
[Jacob Pan]
Signature · 3 lines
Thanks, Jacob
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
Quoted 38 lines · Jacob Pan
Quoted 28 lines · On Fri, 5 Apr 2013 13:23:09 -0700 Greg KH <[email protected]> wrote:
Quoted 27 lines · Jacob Pan
Quoted 24 lines · On Wed, 3 Apr 2013 09:35:09 -0700 Greg KH <[email protected]> wrote:
Quoted 14 lines · Jacob Pan
Quoted 3 lines
Let's step back and start over, what exactly are you trying to tell userspace? What data do you have that you need to express to it? How do you want userspace to see/use it?
It is a good idea to step back and let me explain what I wanted to do here for userspace.
I have two kinds of applications that might use this driver.
1. simple use case where user sets a power limit for a RAPL domain. e.g. set graphics unit power limit to 7w
2. advanced use case where use can do fine tuning on top of simple power limit,e.g. the dynamic response parameters of power control logic, event notifications, etc.Quoted 5 lines
For #1, this driver register with the abstract generic thermal layer (
/sys/class/thermal) and presents itself as a set of cooling devices with a single knob per domain for power limits.root@chromoly:/sys/class/thermal/cooling_device15# echo 7000 > cur_stateGreat, how about submitting that functionality as patch 1 of your series? That seems like a very "normal" thermal driver, right?
Quoted 7 lines
Quoted 2 lines
yes, that would be a normal thermal cooling device driver. I will do that first. Thanks for the suggestion.
Do that first, get it merged, and then let's work on the second part. The patch for that will be much more obvious as to what you are attempting to do.
Quoted 6 lines
Sorry I was too busy to work on v2 before seeing this. I agree I need to simplify the interface, I just need to come up with a more intelligent way to abstract that and do the best guesses for the user. Hopefully, v2 will serve as a confirmation on the comments I got from v1. i.e.
kobject->structdevice, removed dependencies on sysfs internal data struct, etc.
I'm not going to review that part of the code, sorry, as it's about to be ripped out anyway :)
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
Perhaps the thermal interface could be expanded to provide
more functionality that you need?yes, some of them such as limits. But not all the data in the list above are suitable for thermal interface. That is why I am trying to balance between abstracted generic data and RAPL specific data while still allow linking between the two.
Quoted 6 lines
Quoted 2 lines
What is not in the existing interface? And as this is a thermal device, why can't you add them there?
existing interface has only
cur_stateandmax_state, I have been working with Rui (thermal maintainer) on adding more knobs for cooling devices, such as limit low, limit high, event control. I believe they can all be added.
Great, then you will not need any of the driver-specific sysfs files, struct device usage, or kobject mess, so your code should be a lot smaller.
greg k-h
Re: [PATCH 1/1] Introduce Intel RAPL cooling device driver
In reply to Jacob Pandrivers/platform/x86/Kconfig config APPLE_GMUX · line 786
Quoted 9 lines · Jacob Pan
drivers/platform/x86/Kconfig +3
-0@@ -781,4 +781,12 @@ config APPLE_GMUX graphics as well as the backlight. Currently only backlight control is supported by the driver. +config INTEL_RAPL + tristate "Intel RAPL Support" + depends on X86 && THERMAL
drivers/platform/x86/Kconfig is sourced from drivers/platform/Kconfig. The relevant line is wrapped in an "if X86" and "endif" pair. So X86 is superfluous here.
"
drivers/platform/x86/Kconfig config APPLE_GMUX · line 792
Quoted 6 lines
drivers/platform/x86/Kconfig +5
-0+ default y + ---help--- + RAPL, AKA, Running Average Power Limit provides mechanisms to enforce + and monitor per domain power consumption limits of supported Intel CPUs. + endif # X86_PLATFORM_DEVICES
Paul Bolle