Skip to contentSkip to conversation

perf,x86: add Intel RAPL PMU support

PATCH Superseded Stephane Eranian 3 participants 20 messages Oct 23, 2013 – Oct 28, 2013

Subsystems (2)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v3 cover perf,x86: add Intel RAPL PMU support

This patch adds a new uncore PMU to expose the Intel
RAPL (Running Average Power Limit) energy consumption counters. Up to 3 counters, each counting a particular RAPL event are exposed.

The RAPL counters are available on Intel SandyBridge, IvyBridge, Haswell. The server skus add a 3rd counter to measure DRAM power consumption.

The following events are available nd exposed in sysfs:
- power/energy-cores: power consumption of all cores on socket
- power/energy-pkg: power consumption of all cores + LLc cache
- power/energy-dram: power consumption of DRAM (server skus only)

The RAPL PMU is uncore by nature and is implemented such that it only works in system-wide mode. Measuring only one CPU per socket is sufficient. The /sys/devices/rapl/cpumask is exported and can be used by tools to figure out which CPU to monitor by default. For instance, on a 2-socket system, 2 CPUs (one on each socket) will be shown.

The counters all count in the same unit. The perf_events API exposes all RAPL counters as 64-bit integers counting in unit of 1/2^32 Joules (or 0.23 nJ). User level tools must convert the counts by multiplying them by 0.23 and divide 10^9 to obtain Joules. The reason for this is that the kernel avoids doing floating point math whenever possible because it is expensive (user floating-point state must be saved). The method used avoids kernel floating-point and minimizes the loss of precision (bits). Thanks to PeterZ for suggesting this approach.

To convert the raw count in Watt: W = C * 0.23 / (1e9 * time)

The kernel exposes both the scaling factor (0.23 nJ) and the unit (Joules) in sysfs:

$ ls -1 /sys/devices/power/events/energy-*
/sys/devices/power/events/energy-cores
/sys/devices/power/events/energy-cores.scale
/sys/devices/power/events/energy-cores.unit
/sys/devices/power/events/energy-pkg
/sys/devices/power/events/energy-pkg.scale
/sys/devices/power/events/energy-pkg.unit
$ cat /sys/devices/power/events/energy-cores.scale
2.3e-10
$ cat cat /sys/devices/power/events/energy-cores.unit
Joules

RAPL PMU is a new standalone PMU which registers with the perf_event core subsystem. The PMU type (attr->type) is dynamically allocated and is available from /sys/device/rapl/type.

Sampling is not supported by the RAPL PMU. There is no privilege level filtering either.

The PMU exports a cpumask in /sys/devices/power/cpumask. It is used by perf to ensure only one instance of each RAPL event is measured per processor socket. Hotplug CPU is also supported.

The perf stat infrasrtructure is modified to now show event unit. It also applies the scaling factor. As such it will print RAPL events in Joules (and not increments on 0.23 nJ):

 # perf stat -a -e power/energy-pkg/,power/energy-cores/,cycles -I 1000 sleep 1000
 #          time             counts   unit events
     1.000282860               2.51 Joules power/energy-pkg/         [100.00%]
     1.000282860               0.31 Joules power/energy-cores/      
     1.000282860           37765378 ?      cycles                    [100.00%]

The patch adds a hrtimer to poll the counters given that they do no interrupt on overflow. Hardware counters are 32-bit wide.

In v2, we add the locking necesarry to protect the rapl_pmu struct. We also add a description at the top of the file. We check for Intel only processor. We improved the data layout of the rapl_pmu struct. We also lifted the restriction of the number of instances of RAPL counters that can be active at the same time. RAPL is free running counters, so ought to be able to measure events as many times as necessary in parallel via multiple tools. There is never multiplexing among RAPL events.

In v3, we have renamed the event to be more generic power/* instead of rapl/*. We have modified perf stat to print the event with the unit and scaling factors.

Supported CPUs: SandyBridge, IvyBridge, Haswell.

Stephane Eranian (4):
perf: add active_entry list head to struct perf_event perf stat: add event unit and scale support perf,x86: add Intel RAPL PMU support perf,x86: add RAPL hrtimer support

 arch/x86/kernel/cpu/Makefile                |    2 +-
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |  717 +++++++++++++++++++++++++++
 include/linux/perf_event.h                  |    1 +
 kernel/events/core.c                        |    1 +
 tools/perf/builtin-stat.c                   |   72 ++-
 tools/perf/util/evsel.c                     |    2 +
 tools/perf/util/evsel.h                     |    3 +
 tools/perf/util/parse-events.c              |    1 +
 tools/perf/util/pmu.c                       |  170 ++++++-
 tools/perf/util/pmu.h                       |    3 +
 10 files changed, 950 insertions(+), 22 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/perf_event_intel_rapl.c
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH v3 1/4 perf: add active_entry list head to struct perf_event

This patch adds a new fields to the struct perf_event. It is intended to be used to chain events which are active (enabled). It helps in the hardware layer for PMU which do not have actual counter restrictions, i.e., free running read-only counters. Active events are chained as opposed to being tracked via the counter they use.

---
 include/linux/perf_event.h |    1 +
 kernel/events/core.c       |    1 +
 2 files changed, 2 insertions(+)
include/linux/perf_event.h +1 -0
index 2e069d1..a376384 100644
@@ -435,6 +435,7 @@ struct perf_event {
 	struct perf_cgroup		*cgrp; /* cgroup event is attach to */
 	int				cgrp_defer_enabled;
 #endif
+	struct list_head		active_entry;
 
 #endif /* CONFIG_PERF_EVENTS */
 };
kernel/events/core.c +1 -0
index 5bd7fe4..6ef9d19 100644
@@ -6629,6 +6629,7 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	INIT_LIST_HEAD(&event->event_entry);
 	INIT_LIST_HEAD(&event->sibling_list);
 	INIT_LIST_HEAD(&event->rb_entry);
+	INIT_LIST_HEAD(&event->active_entry);
 
 	init_waitqueue_head(&event->waitq);
 	init_irq_work(&event->pending, perf_pending_event);
Signature · 1 line
1.7.9.5
#2 lore raw

Re: [PATCH v3 1/4] perf: add active_entry list head to struct perf_event

Quoted 18 lines · Stephane Eranian

This patch adds a new fields to the struct perf_event. It is intended to be used to chain events which are active (enabled). It helps in the hardware layer for PMU which do not have actual counter restrictions, i.e., free running read-only counters. Active events are chained as opposed to being tracked via the counter they use.

---
 include/linux/perf_event.h |    1 +
 kernel/events/core.c       |    1 +
 2 files changed, 2 insertions(+)
include/linux/perf_event.h +0 -0
index 2e069d1..a376384 100644

include/linux/perf_event.h struct perf_event { · line 438

Quoted 4 lines
include/linux/perf_event.h +1 -0
@@ -435,6 +435,7 @@ struct perf_event {
 	struct perf_cgroup		*cgrp; /* cgroup event is attach to */
 	int				cgrp_defer_enabled;
 #endif
+	struct list_head		active_entry;

Could this be in union with 'hlist_entry' ? It looks as 'same purpose' and 'mutualy exclusive stuff.

jirka

#3 lore raw

Re: [PATCH v3 1/4] perf: add active_entry list head to struct perf_event

Quoted 15 lines · Jiri Olsa
Quoted 13 lines · Stephane Eranian

This patch adds a new fields to the struct perf_event. It is intended to be used to chain events which are active (enabled). It helps in the hardware layer for PMU which do not have actual counter restrictions, i.e., free running read-only counters. Active events are chained as opposed to being tracked via the counter they use.

---
 include/linux/perf_event.h |    1 +
 kernel/events/core.c       |    1 +
 2 files changed, 2 insertions(+)
Quoted 13 lines

include/linux/perf_event.h struct perf_event { · line 438

Quoted 9 lines
include/linux/perf_event.h +1 -0
index 2e069d1..a376384 100644
@@ -435,6 +435,7 @@ struct perf_event {
      struct perf_cgroup              *cgrp; /* cgroup event is attach to */
      int                             cgrp_defer_enabled;
 #endif
+     struct list_head                active_entry;

Could this be in union with 'hlist_entry' ? It looks as 'same purpose' and 'mutualy exclusive stuff.

You're saying that I could use the hlist_entry field because it is currently only used by the sw events in the generic layer. But it seems to be a complicated rcu list for the purpose here.

#4 lore raw

Re: [PATCH v3 1/4] perf: add active_entry list head to struct perf_event

Quoted 29 lines · Stephane Eranian
Quoted 15 lines · Jiri Olsa
Quoted 13 lines · Stephane Eranian

This patch adds a new fields to the struct perf_event. It is intended to be used to chain events which are active (enabled). It helps in the hardware layer for PMU which do not have actual counter restrictions, i.e., free running read-only counters. Active events are chained as opposed to being tracked via the counter they use.

---
 include/linux/perf_event.h |    1 +
 kernel/events/core.c       |    1 +
 2 files changed, 2 insertions(+)
Quoted 13 lines

include/linux/perf_event.h struct perf_event { · line 438

Quoted 9 lines
include/linux/perf_event.h +1 -0
index 2e069d1..a376384 100644
@@ -435,6 +435,7 @@ struct perf_event {
      struct perf_cgroup              *cgrp; /* cgroup event is attach to */
      int                             cgrp_defer_enabled;
 #endif
+     struct list_head                active_entry;

Could this be in union with 'hlist_entry' ? It looks as 'same purpose' and 'mutualy exclusive stuff.

Quoted 3 lines

You're saying that I could use the hlist_entry field because it is currently only used by the sw events in the generic layer. But it seems to be a complicated rcu list for the purpose here.

nope, I just meant saving little space like:

union {
	struct list_head                active_entry;
        struct hlist_node               hlist_entry;
}

just a nitpick

jirka

#5 lore raw

Re: [PATCH v3 1/4] perf: add active_entry list head to struct perf_event

Quoted 34 lines · Jiri Olsa
Quoted 29 lines · Stephane Eranian
Quoted 15 lines · Jiri Olsa
Quoted 13 lines · Stephane Eranian

This patch adds a new fields to the struct perf_event. It is intended to be used to chain events which are active (enabled). It helps in the hardware layer for PMU which do not have actual counter restrictions, i.e., free running read-only counters. Active events are chained as opposed to being tracked via the counter they use.

---
 include/linux/perf_event.h |    1 +
 kernel/events/core.c       |    1 +
 2 files changed, 2 insertions(+)
Quoted 13 lines

include/linux/perf_event.h struct perf_event { · line 438

Quoted 9 lines
include/linux/perf_event.h +1 -0
index 2e069d1..a376384 100644
@@ -435,6 +435,7 @@ struct perf_event {
      struct perf_cgroup              *cgrp; /* cgroup event is attach to */
      int                             cgrp_defer_enabled;
 #endif
+     struct list_head                active_entry;

Could this be in union with 'hlist_entry' ? It looks as 'same purpose' and 'mutualy exclusive stuff.

Quoted 3 lines

You're saying that I could use the hlist_entry field because it is currently only used by the sw events in the generic layer. But it seems to be a complicated rcu list for the purpose here.

Quoted 9 lines

nope, I just meant saving little space like:

union {
        struct list_head                active_entry;
        struct hlist_node               hlist_entry;
}

just a nitpick

But you are relying on the fact that active_entry and hlist_entry can never be used at the same time. You're saying this because *so far* hlist_entry is only used for SW events and what I added is only used by RAPL.

If the goal is the same, then we should add a better description for the field:
"chain the event in the list of active events for a PMU instance on a CPU". Handled by PMU specific code only (not generic code).

#6 lore raw

PATCH v3 2/4 perf stat: add event unit and scale support

In reply to Stephane Eranian

This patch adds perf stat support fo rhandling event units and scales as exported by the kernel.

The kernel can export PMU events actual unit and scaling factor via sysfs:

$ ls -1 /sys/devices/power/events/energy-*
/sys/devices/power/events/energy-cores
/sys/devices/power/events/energy-cores.scale
/sys/devices/power/events/energy-cores.unit
/sys/devices/power/events/energy-pkg
/sys/devices/power/events/energy-pkg.scale
/sys/devices/power/events/energy-pkg.unit
$ cat /sys/devices/power/events/energy-cores.scale
2.3e-10
$ cat cat /sys/devices/power/events/energy-cores.unit
Joules

This patch modifies the pmu event alias code to check for the presence of the .unit and .scale files to load the corresponding values. They are then used by perf stat transparentely:

 # perf stat -a -e power/energy-pkg/,power/energy-cores/,cycles -I 1000 sleep 1000
 #          time             counts   unit events
     1.000214717               3.07 Joules power/energy-pkg/         [100.00%]
     1.000214717               0.53 Joules power/energy-cores/
     1.000214717           12965028      ? cycles                    [100.00%]
     2.000749289               3.01 Joules power/energy-pkg/
     2.000749289               0.52 Joules power/energy-cores/
     2.000749289           15817043      ? cycles
---
 tools/perf/builtin-stat.c      |   72 ++++++++++++-----
 tools/perf/util/evsel.c        |    2 +
 tools/perf/util/evsel.h        |    3 +
 tools/perf/util/parse-events.c |    1 +
 tools/perf/util/pmu.c          |  170 +++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/pmu.h          |    3 +
 6 files changed, 230 insertions(+), 21 deletions(-)
tools/perf/builtin-stat.c +53 -19
index 1a9c95d..43dea3b 100644
@@ -138,6 +138,7 @@ static const char		*post_cmd			= NULL;
 static bool			sync_run			= false;
 static unsigned int		interval			= 0;
 static unsigned int		initial_delay			= 0;
+static unsigned int		unit_width			= 4; /* strlen("unit") */
 static bool			forever				= false;
 static struct timespec		ref_time;
 static struct cpu_map		*aggr_map;
@@ -462,17 +463,17 @@ static void print_interval(void)
 	if (num_print_interval == 0 && !csv_output) {
 		switch (aggr_mode) {
 		case AGGR_SOCKET:
-			fprintf(output, "#           time socket cpus             counts events\n");
+			fprintf(output, "#           time socket cpus             counts  %*s events\n", unit_width, "unit");
 			break;
 		case AGGR_CORE:
-			fprintf(output, "#           time core         cpus             counts events\n");
+			fprintf(output, "#           time core         cpus             counts %*s events\n", unit_width, "unit");
 			break;
 		case AGGR_NONE:
-			fprintf(output, "#           time CPU                 counts events\n");
+			fprintf(output, "#           time CPU                counts %*s events\n", unit_width, "unit");
 			break;
 		case AGGR_GLOBAL:
 		default:
-			fprintf(output, "#           time             counts events\n");
+			fprintf(output, "#           time             counts %*s events\n", unit_width, "unit");
 		}
 	}
 
@@ -517,6 +518,7 @@ static int __run_perf_stat(int argc, const char **argv)
 	unsigned long long t0, t1;
 	struct perf_evsel *counter;
 	struct timespec ts;
+	size_t l;
 	int status = 0;
 	const bool forks = (argc > 0);
 
@@ -566,6 +568,10 @@ static int __run_perf_stat(int argc, const char **argv)
 			return -1;
 		}
 		counter->supported = true;
+
+		l = strlen(counter->unit);
+		if (l > unit_width)
+			unit_width = l;
 	}
 
 	if (perf_evlist__apply_filters(evsel_list)) {
@@ -911,19 +917,32 @@ static void abs_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
 	double total, ratio = 0.0, total2;
 	const char *fmt;
 
-	if (csv_output)
-		fmt = "%.0f%s%s";
-	else if (big_num)
-		fmt = "%'18.0f%s%-25s";
-	else
-		fmt = "%18.0f%s%-25s";
+	if (csv_output) {
+		if (evsel->scale != 1.0)
+			fmt = "%.2f%s%s%s%s";
+		else
+			fmt = "%.0f%s%s%s%s";
+	} else if (big_num)
+		if (evsel->scale != 1.0)
+			fmt = "%'18.2f%s%-*s%s%-25s";
+		else
+			fmt = "%'18.0f%s%-*s%s%-25s";
+	else {
+		if (evsel->scale != 1.0)
+			fmt = "%18.2f%s%-*s%s%-25s";
+		else
+			fmt = "%18.0f%s%-*s%s%-25s";
+	}
 
 	aggr_printout(evsel, cpu, nr);
 
 	if (aggr_mode == AGGR_GLOBAL)
 		cpu = 0;
 
-	fprintf(output, fmt, avg, csv_sep, perf_evsel__name(evsel));
+	if (csv_output)
+		fprintf(output, fmt, avg, csv_sep, evsel->unit, csv_sep, perf_evsel__name(evsel));
+	else
+		fprintf(output, fmt, avg, csv_sep, unit_width, evsel->unit, csv_sep, perf_evsel__name(evsel));
 
 	if (evsel->cgrp)
 		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
@@ -1062,6 +1081,7 @@ static void print_aggr(char *prefix)
 {
 	struct perf_evsel *counter;
 	int cpu, cpu2, s, s2, id, nr;
+	double uval;
 	u64 ena, run, val;
 
 	if (!(aggr_map || aggr_get_id))
@@ -1088,9 +1108,13 @@ static void print_aggr(char *prefix)
 			if (run == 0 || ena == 0) {
 				aggr_printout(counter, id, nr);
 
-				fprintf(output, "%*s%s%*s",
+				fprintf(output, "%*s%s%*s%s%*s",
 					csv_output ? 0 : 18,
 					counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
+
+					csv_sep,
+					csv_output ? 0 : -10,
+					counter->unit,
 					csv_sep,
 					csv_output ? 0 : -24,
 					perf_evsel__name(counter));
@@ -1102,11 +1126,12 @@ static void print_aggr(char *prefix)
 				fputc('\n', output);
 				continue;
 			}
+			uval = val * counter->scale;
 
 			if (nsec_counter(counter))
-				nsec_printout(id, nr, counter, val);
+				nsec_printout(id, nr, counter, uval);
 			else
-				abs_printout(id, nr, counter, val);
+				abs_printout(id, nr, counter, uval);
 
 			if (!csv_output) {
 				print_noise(counter, 1.0);
@@ -1129,6 +1154,7 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	struct perf_stat *ps = counter->priv;
 	double avg = avg_stats(&ps->res_stats[0]);
 	int scaled = counter->counts->scaled;
+	double uval;
 
 	if (prefix)
 		fprintf(output, "%s", prefix);
@@ -1148,10 +1174,12 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 		return;
 	}
 
+	uval = avg * counter->scale;
+
 	if (nsec_counter(counter))
-		nsec_printout(-1, 0, counter, avg);
+		nsec_printout(-1, 0, counter, uval);
 	else
-		abs_printout(-1, 0, counter, avg);
+		abs_printout(-1, 0, counter, uval);
 
 	print_noise(counter, avg);
 
@@ -1178,6 +1206,7 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 static void print_counter(struct perf_evsel *counter, char *prefix)
 {
 	u64 ena, run, val;
+	double uval;
 	int cpu;
 
 	for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
@@ -1189,12 +1218,15 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 			fprintf(output, "%s", prefix);
 
 		if (run == 0 || ena == 0) {
-			fprintf(output, "CPU%*d%s%*s%s%*s",
+			fprintf(output, "CPU%*d%s%*s%s%*s%s%*s",
 				csv_output ? 0 : -4,
 				perf_evsel__cpus(counter)->map[cpu], csv_sep,
 				csv_output ? 0 : 18,
 				counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
 				csv_sep,
+				csv_output ? 0 : -10,
+				counter->unit,
+				csv_sep,
 				csv_output ? 0 : -24,
 				perf_evsel__name(counter));
 
@@ -1206,10 +1238,12 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 			continue;
 		}
 
+		uval = val * counter->scale;
+
 		if (nsec_counter(counter))
-			nsec_printout(cpu, 0, counter, val);
+			nsec_printout(cpu, 0, counter, uval);
 		else
-			abs_printout(cpu, 0, counter, val);
+			abs_printout(cpu, 0, counter, uval);
 
 		if (!csv_output) {
 			print_noise(counter, 1.0);
tools/perf/util/evsel.c +2 -0
index 3a334f0..867971b 100644
@@ -162,6 +162,8 @@ void perf_evsel__init(struct perf_evsel *evsel,
 	evsel->idx	   = idx;
 	evsel->attr	   = *attr;
 	evsel->leader	   = evsel;
+	evsel->unit	   = "?";
+	evsel->scale	   = 1.0;
 	INIT_LIST_HEAD(&evsel->node);
 	hists__init(&evsel->hists);
 	evsel->sample_size = __perf_evsel__sample_size(attr->sample_type);
tools/perf/util/evsel.h +3 -0
index 5aa68cd..d5c6606 100644
@@ -68,6 +68,8 @@ struct perf_evsel {
 	u32			ids;
 	struct hists		hists;
 	char			*name;
+	double			scale;
+	const char		*unit;
 	struct event_format	*tp_format;
 	union {
 		void		*priv;
@@ -130,6 +132,7 @@ extern const char *perf_evsel__sw_names[PERF_COUNT_SW_MAX];
 int __perf_evsel__hw_cache_type_op_res_name(u8 type, u8 op, u8 result,
 					    char *bf, size_t size);
 const char *perf_evsel__name(struct perf_evsel *evsel);
+
 const char *perf_evsel__group_name(struct perf_evsel *evsel);
 int perf_evsel__group_desc(struct perf_evsel *evsel, char *buf, size_t size);
 
tools/perf/util/parse-events.c +1 -0
index c90e55c..be7eba8 100644
@@ -838,6 +838,7 @@ int parse_events_name(struct list_head *list, char *name)
 	list_for_each_entry(evsel, list, node) {
 		if (!evsel->name)
 			evsel->name = strdup(name);
+		pmu_get_event_unit_scale(evsel);
 	}
 
 	return 0;
tools/perf/util/pmu.c +168 -2

222 lines not on this page · raw message

tools/perf/util/pmu.h +3 -0
index 1179b26..6bf23b2 100644
@@ -4,6 +4,7 @@
 #include <linux/bitops.h>
 #include <linux/perf_event.h>
 #include <stdbool.h>
+#include "util/evsel.h"
 
 enum {
 	PERF_PMU_FORMAT_VALUE_CONFIG,
@@ -45,4 +46,6 @@ void print_pmu_events(const char *event_glob, bool name_only);
 bool pmu_have_event(const char *pname, const char *name);
 
 int perf_pmu__test(void);
+
+int pmu_get_event_unit_scale(struct perf_evsel *evsel);
 #endif /* __PMU_H */
Signature · 1 line
1.7.9.5
#7 lore raw

PATCH v3 3/4 perf,x86: add Intel RAPL PMU support

In reply to Stephane Eranian

This patch adds a new uncore PMU to expose the Intel RAPL energy consumption counters. Up to 3 counters, each counting a particular RAPL event are exposed.

The RAPL counters are available on Intel SandyBridge, IvyBridge, Haswell. The server skus add a 3rd counter.

The following events are available and exposed in sysfs:
- power/energy-cores: power consumption of all cores on socket
- power/energy-pkg: power consumption of all cores + LLc cache
- power/energy-dram: power consumption of DRAM (servers only)

For each event both the unit (Joules) and scale (0.23 nJ) is exposed in sysfs for use by perf stat and other tools. Files are:
/sys/devices/power/events/energy-*.unit /sys/devices/power/events/energy-*.scale

The RAPL PMU is uncore by nature and is implemented such that it only works in system-wide mode. Measuring only one CPU per socket is sufficient. The /sys/devices/power/cpumask file can be used by tools to figure out which CPUs
to monitor by default. For instance, on a 2-socket system, 2 CPUs (one on each socket) will be shown.

The counters all count in the same unit (exposed via sysfs). The perf_events API exposes all RAPL counters as 64-bit integers counting in unit of 1/2^32 Joules (or 0.23 nJ). User level tools must convert the counts by multiplying them by 0.23 and divide 10^9 to obtain Joules. The reason for this is that the kernel avoids doing floating point math whenever possible because it is expensive (user floating-point state must be saved). The method used avoids kernel floating-point and minimizes the loss of precision (bits). Thanks to PeterZ for suggesting this approach.

To convert the raw count in Watt:
W = C * 0.23 / (1e9 * time)
or ldexp(C, -32).

RAPL PMU is a new standalone PMU which registers with the perf_event core subsystem. The PMU type (attr->type) is dynamically allocated and is available from /sys/device/power/type.

Sampling is not supported by the RAPL PMU. There is no privilege level filtering either.

---
 arch/x86/kernel/cpu/Makefile                |    2 +-
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |  652 +++++++++++++++++++++++++++
 2 files changed, 653 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kernel/cpu/perf_event_intel_rapl.c
arch/x86/kernel/cpu/Makefile +1 -1
index 47b56a7..6359506 100644
@@ -36,7 +36,7 @@ obj-$(CONFIG_CPU_SUP_AMD)		+= perf_event_amd_iommu.o
 endif
 obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_p6.o perf_event_knc.o perf_event_p4.o
 obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_lbr.o perf_event_intel_ds.o perf_event_intel.o
-obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_uncore.o
+obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_uncore.o perf_event_intel_rapl.o
 endif
 
 
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +652 -0

652 lines not on this page · raw message

Signature · 1 line
1.7.9.5
#8 lore raw

On Wed, Oct 23, 2013 at 02:58:04PM +0200, Stephane Eranian wrote:

SNIP

arch/x86/kernel/cpu/perf_event_intel_rapl.c line 638

Quoted 10 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +10 -0
+
+	perf_cpu_notifier(rapl_cpu_notifier);
+
+	ret = perf_pmu_register(&rapl_pmu_class, "power", -1);
+	WARN_ON(ret);
+	if (!ret) {
+		pr_info("RAPL PMU detected, registration failed, RAPL PMU disabled\n");
+		put_online_cpus();
+		return -1;
+	}

should above rather be:

        if (WARN_ON(ret)) {
                pr_info("RAPL PMU detected, registration failed, RAPL PMU disabled\n");

jirka

#9 lore raw
In reply to Stephane Eranian

On Wed, Oct 23, 2013 at 02:58:04PM +0200, Stephane Eranian wrote:

SNIP

arch/x86/kernel/cpu/perf_event_intel_rapl.c line 630

Quoted 12 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +12 -0
+			pmu = per_cpu(rapl_pmu, i);
+			if (pmu) {
+				per_cpu(rapl_pmu, cpu) = pmu;
+				atomic_inc(&pmu->refcnt);
+			}
+			continue;
+		}
+		rapl_cpu_prepare(cpu);
+		cpumask_set_cpu(cpu, &rapl_cpu_mask);
+	}
+
+	perf_cpu_notifier(rapl_cpu_notifier);

hum, this should be rather called below only if we succeed with the perf_pmu_register

Quoted 25 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +22 -0
+
+	ret = perf_pmu_register(&rapl_pmu_class, "power", -1);
+	WARN_ON(ret);
+	if (!ret) {
+		pr_info("RAPL PMU detected, registration failed, RAPL PMU disabled\n");
+		put_online_cpus();
+		return -1;
+	}
+
+	pmu = __get_cpu_var(rapl_pmu);
+
+	pr_info("RAPL PMU detected, hw unit 2^-%d Joules,"
+		" API unit is 2^-32 Joules,"
+		" %d fixed counters\n",
+		pmu->hw_unit,
+		hweight32(rapl_cntr_mask));
+
+	put_online_cpus();
+
+	return 0;
+}
+device_initcall(rapl_pmu_init);
Signature · 1 line
1.7.9.5
#10 lore raw
In reply to Stephane Eranian

On Wed, Oct 23, 2013 at 02:58:04PM +0200, Stephane Eranian wrote:

SNIP

arch/x86/kernel/cpu/perf_event_intel_rapl.c line 430

Quoted 11 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +11 -0
+
+static void rapl_init_cpu(int cpu)
+{
+	int i, phys_id = topology_physical_package_id(cpu);
+
+	spin_lock(&rapl_hotplug_lock);
+
+	/* check if phys_is is already covered */
+	for_each_cpu(i, &rapl_cpu_mask) {
+		if (phys_id == topology_physical_package_id(i))
+			return;

missing 'spin_unlock(&rapl_hotplug_lock)' above

Quoted 7 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +7 -0
+	}
+	/* was not found, so add it */
+	cpumask_set_cpu(cpu, &rapl_cpu_mask);
+
+	spin_unlock(&rapl_hotplug_lock);
+}
+
#11 lore raw
Quoted 4 lines · Jiri Olsa

On Wed, Oct 23, 2013 at 02:58:04PM +0200, Stephane Eranian wrote:

SNIP

Quoted 14 lines

arch/x86/kernel/cpu/perf_event_intel_rapl.c line 430

Quoted 11 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +11 -0
+
+static void rapl_init_cpu(int cpu)
+{
+     int i, phys_id = topology_physical_package_id(cpu);
+
+     spin_lock(&rapl_hotplug_lock);
+
+     /* check if phys_is is already covered */
+     for_each_cpu(i, &rapl_cpu_mask) {
+             if (phys_id == topology_physical_package_id(i))
+                     return;

missing 'spin_unlock(&rapl_hotplug_lock)' above

Good catch. I fixed that now.

Quoted 7 lines
Quoted 7 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +7 -0
+     }
+     /* was not found, so add it */
+     cpumask_set_cpu(cpu, &rapl_cpu_mask);
+
+     spin_unlock(&rapl_hotplug_lock);
+}
+
#12 lore raw

Hi,

I was thinking about the scaling issue over the week-end.

We agreed that it was necessary to export the scaling via sysfs per event. The RAPL v3 has an implementation of that including in the perf tool.

If we have that, then it may not be necessary anymore to express the raw count in the 1/2^32 J unit like we are currently doing. This loses a bit of precision. We could as well expose the actual raw count and export the actual unit via sysfs. For instance, on SNB/IVB the unit is 1/2^16, but on Haswell it is 1/2^14.

I see two issues with that approach though:

- the interpretation of the raw count changes from machine to machine and needs to ALWAYS be used with the scaling. It is not always possible to just compare raw counts directly.

- we would need a way to express that ratio without actually calculating it in the kernel. There are 6 ratios possible. So we either have a lookup table with the floating point values already computed and encoded as strings. Or we need to add a calculator style parsing in the perf tool (or any other) tool to parse a ratio: 1 / 65536 or any basic mathematical expression. After all, the scaling support has to be generic. Other events may use a different form of scaling ratios. But that seems overkill to me.

So in the end, it may be the case that what we have not in RAPLv3 is the simplest approach.

Any opinion?

Quoted 28 lines · Stephane Eranian
Quoted 4 lines · Jiri Olsa

On Wed, Oct 23, 2013 at 02:58:04PM +0200, Stephane Eranian wrote:

SNIP

Quoted 14 lines

arch/x86/kernel/cpu/perf_event_intel_rapl.c line 430

Quoted 11 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +11 -0
+
+static void rapl_init_cpu(int cpu)
+{
+     int i, phys_id = topology_physical_package_id(cpu);
+
+     spin_lock(&rapl_hotplug_lock);
+
+     /* check if phys_is is already covered */
+     for_each_cpu(i, &rapl_cpu_mask) {
+             if (phys_id == topology_physical_package_id(i))
+                     return;

missing 'spin_unlock(&rapl_hotplug_lock)' above

Good catch. I fixed that now.

Quoted 7 lines
Quoted 7 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +7 -0
+     }
+     /* was not found, so add it */
+     cpumask_set_cpu(cpu, &rapl_cpu_mask);
+
+     spin_unlock(&rapl_hotplug_lock);
+}
+
maintainer
#13 lore raw
Quoted 6 lines · Stephane Eranian

If we have that, then it may not be necessary anymore to express the raw count in the 1/2^32 J unit like we are currently doing. This loses a bit of precision. We could as well expose the actual raw count and export the actual unit via sysfs. For instance, on SNB/IVB the unit is 1/2^16, but on Haswell it is 1/2^14.

2^-32 can losslessly express both 2^-16 and 2^-14.

Notably: 2^18/2^32 = 2^(18-32) = 2^-14.

So no, 2^-32 does not loose precision.

The only side effect of always using 2^-32 is that we can only maximally represent 2^32 (from 64-32), whereas when using 2^-14 we could maximally represent 2^50.

That said; 2^32 Joule ~ 4.2 GJ which is a rather large quantity of energy; one I would hope is out there when measuring package energy costs over any reasonable amount of time.

So the only reason to switch away from using the 32.32 fixed point would be if someone can make a reasonable argument for why 4.2 GJ is not sufficient and they need 1 PJ (yes, peta-joule, as in we need a private nuclear reactor to power this CPU).

#14 lore raw

Peter,

Quoted 8 lines · Peter Zijlstra
Quoted 6 lines · Stephane Eranian

If we have that, then it may not be necessary anymore to express the raw count in the 1/2^32 J unit like we are currently doing. This loses a bit of precision. We could as well expose the actual raw count and export the actual unit via sysfs. For instance, on SNB/IVB the unit is 1/2^16, but on Haswell it is 1/2^14.

Quoted 6 lines

2^-32 can losslessly express both 2^-16 and 2^-14.

Notably: 2^18/2^32 = 2^(18-32) = 2^-14.

So no, 2^-32 does not loose precision.

You are correct. No bits are lost.

Quoted 4 lines

The only side effect of always using 2^-32 is that we can only maximally represent 2^32 (from 64-32), whereas when using 2^-14 we could maximally represent 2^50.

Quoted 8 lines

That said; 2^32 Joule ~ 4.2 GJ which is a rather large quantity of energy; one I would hope is out there when measuring package energy costs over any reasonable amount of time.

So the only reason to switch away from using the 32.32 fixed point would be if someone can make a reasonable argument for why 4.2 GJ is not sufficient and they need 1 PJ (yes, peta-joule, as in we need a private nuclear reactor to power this CPU).

I think we are fine with what we have. Simple, no precision lost, constant user-visible scaling factor easy to export as a string to user tools. Comparison of raw count possible directly.

I will post v4 very soon.
Thanks.

#15 lore raw

PATCH v3 4/4 perf,x86: add RAPL hrtimer support

In reply to Stephane Eranian

The RAPL PMU counters do not interrupt on overflow. Therefore, the kernel needs to poll the counters to avoid missing an overflow. This patch adds the hrtimer code to do this.

The timer internval is calculated at boot time based on the power unit used by the HW.

---
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |   75 +++++++++++++++++++++++++--
 1 file changed, 70 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_rapl.c +70 -5
index 3d71d39..ed0566a 100644
@@ -92,11 +92,13 @@ static struct kobj_attribute format_attr_##_var =		\
 
 struct rapl_pmu {
 	spinlock_t	lock;
-	atomic_t	refcnt;
 	int		hw_unit;  /* 1/2^hw_unit Joule */
-	int		phys_id;
-	int		n_active; /* number of active events */
+	struct hrtimer	hrtimer;
 	struct list_head active_list;
+	ktime_t		timer_interval; /* in ktime_t unit */
+	int		n_active; /* number of active events */
+	int		phys_id;
+	atomic_t	refcnt;
 };
 
 static struct pmu rapl_pmu_class;
@@ -161,6 +163,47 @@ static u64 rapl_event_update(struct perf_event *event)
 	return new_raw_count;
 }
 
+static void rapl_start_hrtimer(struct rapl_pmu *pmu)
+{
+	__hrtimer_start_range_ns(&pmu->hrtimer,
+			pmu->timer_interval, 0,
+			HRTIMER_MODE_REL_PINNED, 0);
+}
+
+static void rapl_stop_hrtimer(struct rapl_pmu *pmu)
+{
+	hrtimer_cancel(&pmu->hrtimer);
+}
+
+static enum hrtimer_restart rapl_hrtimer_handle(struct hrtimer *hrtimer)
+{
+	struct rapl_pmu *pmu = container_of(hrtimer, struct rapl_pmu, hrtimer);
+	struct perf_event *event;
+	unsigned long flags;
+
+	if (!pmu->n_active)
+		return HRTIMER_NORESTART;
+
+	spin_lock_irqsave(&pmu->lock, flags);
+
+	list_for_each_entry(event, &pmu->active_list, active_entry) {
+		rapl_event_update(event);
+	}
+
+	spin_unlock_irqrestore(&pmu->lock, flags);
+
+	hrtimer_forward_now(&pmu->hrtimer, pmu->timer_interval);
+
+	return HRTIMER_RESTART;
+}
+
+static void rapl_hrtimer_init(struct rapl_pmu *pmu)
+{
+	hrtimer_init(&pmu->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	pmu->hrtimer.function = rapl_hrtimer_handle;
+}
+
+
 static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 				   struct perf_event *event)
 {
@@ -174,6 +217,8 @@ static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 	local64_set(&event->hw.prev_count, rapl_read_counter(event));
 
 	pmu->n_active++;
+	if (pmu->n_active == 1)
+		rapl_start_hrtimer(pmu);
 }
 
 static void rapl_pmu_event_start(struct perf_event *event, int mode)
@@ -198,6 +243,8 @@ static void rapl_pmu_event_stop(struct perf_event *event, int mode)
 	if (!(hwc->state & PERF_HES_STOPPED)) {
 		WARN_ON_ONCE(pmu->n_active <= 0);
 		pmu->n_active--;
+		if (pmu->n_active == 0)
+			rapl_stop_hrtimer(pmu);
 
 		list_del(&event->active_entry);
 
@@ -439,6 +486,7 @@ static int rapl_cpu_prepare(int cpu)
 {
 	struct rapl_pmu *pmu = per_cpu(rapl_pmu, cpu);
 	int phys_id = topology_physical_package_id(cpu);
+	u64 ms;
 
 	if (pmu)
 		return 0;
@@ -464,6 +512,20 @@ static int rapl_cpu_prepare(int cpu)
 	rdmsrl(MSR_RAPL_POWER_UNIT, pmu->hw_unit);
 	pmu->hw_unit = (pmu->hw_unit >> 8) & 0x1FULL;
 
+	/*
+	 * use reference of 200W for scaling the timeout
+	 * to avoid missing counter overflows.
+	 * 200W = 200 Joules/sec
+	 * divide interval by 2 to avoid lockstep (2 * 100)
+	 * if hw unit is 32, then we use 2 ms 1/200/2
+	 */
+	if (pmu->hw_unit < 32)
+		ms = 1000 * (1ULL << (32 - pmu->hw_unit - 1)) / (2 * 100);
+	else
+		ms = 2;
+
+	pmu->timer_interval = ms_to_ktime(ms);
+
 	/* set RAPL pmu for this cpu for now */
 	per_cpu(rapl_pmu_kfree, cpu) = NULL;
 	per_cpu(rapl_pmu, cpu) = pmu;
@@ -625,6 +687,7 @@ static int __init rapl_pmu_init(void)
 		}
 		rapl_cpu_prepare(cpu);
 		cpumask_set_cpu(cpu, &rapl_cpu_mask);
+		rapl_hrtimer_init(per_cpu(rapl_pmu, cpu));
 	}
 
 	perf_cpu_notifier(rapl_cpu_notifier);
@@ -641,9 +704,11 @@ static int __init rapl_pmu_init(void)
 
 	pr_info("RAPL PMU detected, hw unit 2^-%d Joules,"
 		" API unit is 2^-32 Joules,"
-		" %d fixed counters\n",
+		" %d fixed counters"
+		" %llu ms ovfl timer\n",
 		pmu->hw_unit,
-		hweight32(rapl_cntr_mask));
+		hweight32(rapl_cntr_mask),
+		ktime_to_ms(pmu->timer_interval));
 
 	put_online_cpus();
 
Signature · 1 line
1.7.9.5
#16 lore raw

Re: [PATCH v3 4/4] perf,x86: add RAPL hrtimer support

Quoted 54 lines · Stephane Eranian

The RAPL PMU counters do not interrupt on overflow. Therefore, the kernel needs to poll the counters to avoid missing an overflow. This patch adds the hrtimer code to do this.

The timer internval is calculated at boot time based on the power unit used by the HW.

---
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |   75 +++++++++++++++++++++++++--
 1 file changed, 70 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_rapl.c +21 -3
index 3d71d39..ed0566a 100644
@@ -92,11 +92,13 @@ static struct kobj_attribute format_attr_##_var =		\
 
 struct rapl_pmu {
 	spinlock_t	lock;
-	atomic_t	refcnt;
 	int		hw_unit;  /* 1/2^hw_unit Joule */
-	int		phys_id;
-	int		n_active; /* number of active events */
+	struct hrtimer	hrtimer;
 	struct list_head active_list;
+	ktime_t		timer_interval; /* in ktime_t unit */
+	int		n_active; /* number of active events */
+	int		phys_id;
+	atomic_t	refcnt;
 };
 
 static struct pmu rapl_pmu_class;
@@ -161,6 +163,47 @@ static u64 rapl_event_update(struct perf_event *event)
 	return new_raw_count;
 }
 
+static void rapl_start_hrtimer(struct rapl_pmu *pmu)
+{
+	__hrtimer_start_range_ns(&pmu->hrtimer,
+			pmu->timer_interval, 0,
+			HRTIMER_MODE_REL_PINNED, 0);
+}
+
+static void rapl_stop_hrtimer(struct rapl_pmu *pmu)
+{
+	hrtimer_cancel(&pmu->hrtimer);
+}
+
+static enum hrtimer_restart rapl_hrtimer_handle(struct hrtimer *hrtimer)
+{
+	struct rapl_pmu *pmu = container_of(hrtimer, struct rapl_pmu, hrtimer);
+	struct perf_event *event;

arch/x86/kernel/cpu/perf_event_intel_rapl.c static u64 rapl_event_update(struct perf_event *event) · line 191

Quoted 10 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c +10 -0
+	unsigned long flags;
+
+	if (!pmu->n_active)
+		return HRTIMER_NORESTART;
+
+	spin_lock_irqsave(&pmu->lock, flags);
+
+	list_for_each_entry(event, &pmu->active_list, active_entry) {
+		rapl_event_update(event);
+	}

hi,
I dont fully understand the reason for the timer,
I'm probably missing something..

- the timer calls rapl_event_update for all defined events
- but rapl_pmu_event_read calls rapl_event_update any time the event is read (sys_read)

The rapl_event_update only read msr and updates
event->count|hw,prev_count.

What's the timer purpose then?

thanks for info,
jirka

#17 lore raw

Re: [PATCH v3 4/4] perf,x86: add RAPL hrtimer support

Quoted 61 lines · Jiri Olsa
Quoted 60 lines · Stephane Eranian

The RAPL PMU counters do not interrupt on overflow. Therefore, the kernel needs to poll the counters to avoid missing an overflow. This patch adds the hrtimer code to do this.

The timer internval is calculated at boot time based on the power unit used by the HW.

---
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |   75 +++++++++++++++++++++++++--
 1 file changed, 70 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_rapl.c +27 -3
index 3d71d39..ed0566a 100644
@@ -92,11 +92,13 @@ static struct kobj_attribute format_attr_##_var =         \
 
 struct rapl_pmu {
      spinlock_t      lock;
-     atomic_t        refcnt;
      int             hw_unit;  /* 1/2^hw_unit Joule */
-     int             phys_id;
-     int             n_active; /* number of active events */
+     struct hrtimer  hrtimer;
      struct list_head active_list;
+     ktime_t         timer_interval; /* in ktime_t unit */
+     int             n_active; /* number of active events */
+     int             phys_id;
+     atomic_t        refcnt;
 };
 
 static struct pmu rapl_pmu_class;
@@ -161,6 +163,47 @@ static u64 rapl_event_update(struct perf_event *event)
      return new_raw_count;
 }
 
+static void rapl_start_hrtimer(struct rapl_pmu *pmu)
+{
+     __hrtimer_start_range_ns(&pmu->hrtimer,
+                     pmu->timer_interval, 0,
+                     HRTIMER_MODE_REL_PINNED, 0);
+}
+
+static void rapl_stop_hrtimer(struct rapl_pmu *pmu)
+{
+     hrtimer_cancel(&pmu->hrtimer);
+}
+
+static enum hrtimer_restart rapl_hrtimer_handle(struct hrtimer *hrtimer)
+{
+     struct rapl_pmu *pmu = container_of(hrtimer, struct rapl_pmu, hrtimer);
+     struct perf_event *event;
+     unsigned long flags;
+
+     if (!pmu->n_active)
+             return HRTIMER_NORESTART;
+
+     spin_lock_irqsave(&pmu->lock, flags);
Quoted 9 lines

arch/x86/kernel/cpu/perf_event_intel_rapl.c static u64 rapl_event_update(struct perf_event *event) · line 191

Quoted 4 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c +4 -0
+
+     list_for_each_entry(event, &pmu->active_list, active_entry) {
+             rapl_event_update(event);
+     }

hi,
I dont fully understand the reason for the timer,
I'm probably missing something..

The reason is rather simple and is similar to what happens with uncore. The counter are narrow, 32-bit and there is no interrupt capability. We need to poll the counters and accumulate in the sw counter to avoid missing an overflow.

Quoted 1 line

- the timer calls rapl_event_update for all defined events

No, only for the defined RAPL events which is what we want.

Quoted 3 lines

- but rapl_pmu_event_read calls rapl_event_update any time the event is read (sys_read)

Yes, but we want to prevent missing a counter overflow. It may happen if the counter counts in a unit which increments fast.

Quoted 2 lines

The rapl_event_update only read msr and updates
event->count|hw,prev_count.

No, it does update the count:

        local64_add(sdelta, &event->count);
#18 lore raw

Re: [PATCH v3 4/4] perf,x86: add RAPL hrtimer support

Quoted 2 lines · Stephane Eranian
Quoted 1 line · Jiri Olsa

On Wed, Oct 23, 2013 at 02:58:05PM +0200, Stephane Eranian wrote:

SNIP

Quoted 8 lines
Quoted 8 lines

arch/x86/kernel/cpu/perf_event_intel_rapl.c static u64 rapl_event_update(struct perf_event *event) · line 191

Quoted 3 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c +3 -0
+     list_for_each_entry(event, &pmu->active_list, active_entry) {
+             rapl_event_update(event);
+     }

hi,
I dont fully understand the reason for the timer,
I'm probably missing something..

Quoted 8 lines

The reason is rather simple and is similar to what happens with uncore. The counter are narrow, 32-bit and there is no interrupt capability. We need to poll the counters and accumulate in the sw counter to avoid missing an overflow.

Quoted 1 line

- the timer calls rapl_event_update for all defined events

No, only for the defined RAPL events which is what we want.

ok, that's what I meant

Quoted 11 lines
Quoted 4 lines

- but rapl_pmu_event_read calls rapl_event_update any time the event is read (sys_read)

Yes, but we want to prevent missing a counter overflow. It may happen if the counter counts in a unit which increments fast.

Quoted 2 lines

The rapl_event_update only read msr and updates
event->count|hw,prev_count.

No, it does update the count:

        local64_add(sdelta, &event->count);

ah, there's the shift that takes care of the
overflowed msr value.. ok

thanks,
jirka

#19 lore raw

Re: [PATCH v3 4/4] perf,x86: add RAPL hrtimer support

Quoted 32 lines · Jiri Olsa
Quoted 2 lines · Stephane Eranian
Quoted 1 line · Jiri Olsa

On Wed, Oct 23, 2013 at 02:58:05PM +0200, Stephane Eranian wrote:

SNIP

Quoted 8 lines
Quoted 8 lines

arch/x86/kernel/cpu/perf_event_intel_rapl.c static u64 rapl_event_update(struct perf_event *event) · line 191

Quoted 3 lines
arch/x86/kernel/cpu/perf_event_intel_rapl.c +3 -0
+     list_for_each_entry(event, &pmu->active_list, active_entry) {
+             rapl_event_update(event);
+     }

hi,
I dont fully understand the reason for the timer,
I'm probably missing something..

Quoted 8 lines

The reason is rather simple and is similar to what happens with uncore. The counter are narrow, 32-bit and there is no interrupt capability. We need to poll the counters and accumulate in the sw counter to avoid missing an overflow.

Quoted 1 line

- the timer calls rapl_event_update for all defined events

No, only for the defined RAPL events which is what we want.

ok, that's what I meant

Quoted 7 lines
Quoted 4 lines

- but rapl_pmu_event_read calls rapl_event_update any time the event is read (sys_read)

Yes, but we want to prevent missing a counter overflow. It may happen if the counter counts in a unit which increments fast.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

The rapl_event_update only read msr and updates
event->count|hw,prev_count.

No, it does update the count:

        local64_add(sdelta, &event->count);

ah, there's the shift that takes care of the
overflowed msr value.. ok

No, we try to poll the counter faster than it can possibly overflow.