Skip to contentSkip to conversation

perf/x86: add Intel RAPL PMU support

PATCH In review Stephane Eranian 7 participants 22 messages Nov 12, 2013 – Nov 30, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v7 cover perf/x86: add Intel RAPL PMU support

This patch adds a new uncore PMU to expose the Intel RAPL (Running Average Power Limit) energy consumption counters. Up to 3 counters, each counting a single RAPL event are exposed.

The RAPL counters are available on Intel SandyBridge, IvyBridge, Haswell. The server processors add a 3rd counter to measure DRAM power consumption.

The following events are available and exposed in sysfs:
- power/energy-cores: power consumption of all cores on socket
- power/energy-pkg : power consumption of all cores + LLC cache
- power/energy-dram : power consumption of DRAM (servers only)

The RAPL PMU is uncore by nature and is implemented such that it only works in system-wide mode. Measuring only one CPU per socket is sufficient.

The counters all count in the same unit. The perf_events API exposes all RAPL counters as 64-bit integers counting in unit of 1/2^32 Joules (about 0.23 nJ). User level tools must convert the counts by multiplying them by the scaling factor exposed in the correponding event .scale file in sysfs to obtain a value expressed in Joules. The reason for this approach is that the kernel avoids doing floating point math whenever possible because it is expensive (user floating-point state must be saved). The method used avoids kernel floating-point and does not incur any precision loss. Thanks to PeterZ for suggesting this approach.

To convert the raw count in Watts: W = C * 2.3 / (1e10 * time)

The kernel exposes both the scaling factor and the unit (Joules) in sysfs:

$ ls -1 /sys/devices/power/events/energy-*
/sys/devices/power/events/energy-cores
/sys/devices/power/events/energy-cores.scale
/sys/devices/power/events/energy-cores.unit
/sys/devices/power/events/energy-pkg
/sys/devices/power/events/energy-pkg.scale
/sys/devices/power/events/energy-pkg.unit
$ cat /sys/devices/power/events/energy-cores.scale
2.3283064365386962890625e-10
$ cat cat /sys/devices/power/events/energy-cores.unit
Joules

RAPL PMU is a new standalone PMU which registers with the perf_event core subsystem. The PMU type (attr->type) is dynamically allocated and is available from /sys/device/rapl/type.

Sampling is not supported by the RAPL PMU. There is no privilege level filtering either.

The PMU exports a cpumask in /sys/devices/power/cpumask. It is used by perf to ensure only one instance of each RAPL event is measured per processor socket. Hotplug CPU is also supported.

The perf stat infrastrructure is enhanced to show events units. It also applies the scaling factor. As such, perf stat prints RAPL events in Joules (and not increments of 0.23 nJ):

 # perf stat -a -e power/energy-pkg/,power/energy-cores/,cycles -I 1000 sleep 1000
 #          time             counts   unit events
     1.000282860               2.51 Joules power/energy-pkg/
     1.000282860               0.31 Joules power/energy-cores/
     1.000282860           37765378        cycles

The patch adds a hrtimer to poll the counters given that they do no interrupt on overflow. Hardware counters are 32-bit wide.

In v2, we add the locking necesarry to protect the rapl_pmu struct. We also add a description at the top of the file. We check for Intel only processor. We improved the data layout of the rapl_pmu struct. We also lifted the restriction of the number of instances of RAPL counters that can be active at the same time. RAPL is free running counters, so ought to be able to measure events as many times as necessary in parallel via multiple tools. There is never multiplexing among RAPL events.

In v3, we have renamed the event to be more generic power/* instead of rapl/*. We have modified perf stat to print the event with the unit and scaling factors.

In v4, we integrate the feedback from Jiri and rebase to 3.12-rc7+ from tip.git.

In v5, we export the full scaling factor to increase prescision. In the perf tool, we changed the way the .unit and .scale syfs entries are parsed. Thank to Jiri for this contribution on this. We also fix a couple of printf() issues with perf stat and units. Now, we print no unit symbol when the event has no unit (was ? before). Patch is relative to 3.12 from tip.git.

In v6, we fixed a few issues in the perf tool having to do with printing of the unit. Works for uncore events now. There is a major restructuring of the code in the kernel because the hrtimer needs to be per-cpu and not shared per socket because there can be multiple sessions in parallel and also because of hotplug CPU. The hotplug cpu code was also updated.

In v7, we rebased to 3.12+ and dropped the hotplug cpu lock because it was not needed. Hotplug CPU is still serialized.

Thanks to all contributors to this patch series: PeterZ, Jiri, Maria, Arnaldo, Andi, Ingo.

Supported CPUs: SandyBridge, IvyBridge, Haswell.

Stephane Eranian (4):
perf: add active_entry list head to struct perf_event perf stat: add event unit and scale support perf,x86: add Intel RAPL PMU support perf,x86: add RAPL hrtimer support

 arch/x86/kernel/cpu/Makefile                |    2 +-
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |  661 +++++++++++++++++++++++++++
 include/linux/perf_event.h                  |    5 +-
 kernel/events/core.c                        |    1 +
 tools/perf/builtin-stat.c                   |  114 +++--
 tools/perf/util/evsel.c                     |    2 +
 tools/perf/util/evsel.h                     |    3 +
 tools/perf/util/parse-events.c              |   28 +-
 tools/perf/util/pmu.c                       |  138 +++++-
 tools/perf/util/pmu.h                       |    3 +-
 10 files changed, 911 insertions(+), 46 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/perf_event_intel_rapl.c
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH v7 1/4 perf: add active_entry list head to struct perf_event

This patch adds a new field to the struct perf_event. It is intended to be used to chain events which are active (enabled). It helps in the hardware layer for PMUs which do not have actual counter restrictions, i.e., free running read-only counters. Active events are chained as opposed to being tracked via the counter they use.

To save space we use a union with hlist_entry as both are mutually exclusive (suggested by Jiri Olsa).

---
 include/linux/perf_event.h |    5 ++++-
 kernel/events/core.c       |    1 +
 2 files changed, 5 insertions(+), 1 deletion(-)
include/linux/perf_event.h +4 -1
index 2e069d1..8f4a70f 100644
@@ -319,7 +319,10 @@ struct perf_event {
 	 */
 	struct list_head		migrate_entry;
 
-	struct hlist_node		hlist_entry;
+	union {
+		struct hlist_node	hlist_entry;
+		struct list_head	active_entry;
+	};
 	int				nr_siblings;
 	int				group_flags;
 	struct perf_event		*group_leader;
kernel/events/core.c +1 -0
index 4dc078d..90dca5c 100644
@@ -6663,6 +6663,7 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	INIT_LIST_HEAD(&event->event_entry);
 	INIT_LIST_HEAD(&event->sibling_list);
 	INIT_LIST_HEAD(&event->rb_entry);
+	INIT_LIST_HEAD(&event->active_entry);
 
 	init_waitqueue_head(&event->waitq);
 	init_irq_work(&event->pending, perf_pending_event);
Signature · 1 line
1.7.9.5
#2 lore raw

[tip:perf/core] perf: Add active_entry list head to struct perf_event

perf: Add active_entry list head to struct perf_event
Commit-ID:  71ad88efebbcde374bddf904b96f3a7fc82d45d4
Gitweb:     http://git.kernel.org/tip/71ad88efebbcde374bddf904b96f3a7fc82d45d4
Author:     Stephane Eranian <[email protected]>
AuthorDate: Tue, 12 Nov 2013 17:58:48 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 27 Nov 2013 11:16:38 +0100

perf: Add active_entry list head to struct perf_event

This patch adds a new field to the struct perf_event. It is intended to be used to chain events which are active (enabled). It helps in the hardware layer for PMUs which do not have actual counter restrictions, i.e., free running read-only counters. Active events are chained as opposed to being tracked via the counter they use.

To save space we use a union with hlist_entry as both are mutually exclusive (suggested by Jiri Olsa).

---
 include/linux/perf_event.h | 5 ++++-
 kernel/events/core.c       | 1 +
 2 files changed, 5 insertions(+), 1 deletion(-)
include/linux/perf_event.h +4 -1
index 2e069d1..8f4a70f 100644
@@ -319,7 +319,10 @@ struct perf_event {
 	 */
 	struct list_head		migrate_entry;
 
-	struct hlist_node		hlist_entry;
+	union {
+		struct hlist_node	hlist_entry;
+		struct list_head	active_entry;
+	};
 	int				nr_siblings;
 	int				group_flags;
 	struct perf_event		*group_leader;
kernel/events/core.c +1 -0
index 72348dc..403b781 100644
@@ -6655,6 +6655,7 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	INIT_LIST_HEAD(&event->event_entry);
 	INIT_LIST_HEAD(&event->sibling_list);
 	INIT_LIST_HEAD(&event->rb_entry);
+	INIT_LIST_HEAD(&event->active_entry);
 
 	init_waitqueue_head(&event->waitq);
 	init_irq_work(&event->pending, perf_pending_event);
#3 lore raw

PATCH v7 2/4 perf stat: add event unit and scale support

In reply to Stephane Eranian

This patch adds perf stat support for handling event units and scales as exported by the kernel.

The kernel can export PMU events actual unit and scaling factor via sysfs:

$ ls -1 /sys/devices/power/events/energy-*
/sys/devices/power/events/energy-cores
/sys/devices/power/events/energy-cores.scale
/sys/devices/power/events/energy-cores.unit
/sys/devices/power/events/energy-pkg
/sys/devices/power/events/energy-pkg.scale
/sys/devices/power/events/energy-pkg.unit
$ cat /sys/devices/power/events/energy-cores.scale
2.3283064365386962890625e-10
$ cat cat /sys/devices/power/events/energy-cores.unit
Joules

This patch modifies the pmu event alias code to check for the presence of the .unit and .scale files to load the corresponding values. They are then used by perf stat transparently:

 # perf stat -a -e power/energy-pkg/,power/energy-cores/,cycles -I 1000 sleep 1000
 #          time             counts   unit events
     1.000214717               3.07 Joules power/energy-pkg/         [100.00%]
     1.000214717               0.53 Joules power/energy-cores/
     1.000214717           12965028        cycles                    [100.00%]
     2.000749289               3.01 Joules power/energy-pkg/
     2.000749289               0.52 Joules power/energy-cores/
     2.000749289           15817043        cycles

When the event does not have an explicit unit exported by the kernel, nothing is printed. In csv output mode, there will be an empty field.

Special thanks to Jiri for providing the supporting code in the parser to trigger reading of the scale and unit files.

---
 tools/perf/builtin-stat.c      |  114 ++++++++++++++++++++++++---------
 tools/perf/util/evsel.c        |    2 +
 tools/perf/util/evsel.h        |    3 +
 tools/perf/util/parse-events.c |   28 +++++---
 tools/perf/util/pmu.c          |  138 ++++++++++++++++++++++++++++++++++++++--
 tools/perf/util/pmu.h          |    3 +-
 6 files changed, 244 insertions(+), 44 deletions(-)
tools/perf/builtin-stat.c +84 -30
index 0fc1c94..66d20c9 100644
@@ -138,6 +138,7 @@ static const char		*post_cmd			= NULL;
 static bool			sync_run			= false;
 static unsigned int		interval			= 0;
 static unsigned int		initial_delay			= 0;
+static unsigned int		unit_width			= 4; /* strlen("unit") */
 static bool			forever				= false;
 static struct timespec		ref_time;
 static struct cpu_map		*aggr_map;
@@ -462,17 +463,17 @@ static void print_interval(void)
 	if (num_print_interval == 0 && !csv_output) {
 		switch (aggr_mode) {
 		case AGGR_SOCKET:
-			fprintf(output, "#           time socket cpus             counts events\n");
+			fprintf(output, "#           time socket cpus             counts %*s events\n", unit_width, "unit");
 			break;
 		case AGGR_CORE:
-			fprintf(output, "#           time core         cpus             counts events\n");
+			fprintf(output, "#           time core         cpus             counts %*s events\n", unit_width, "unit");
 			break;
 		case AGGR_NONE:
-			fprintf(output, "#           time CPU                 counts events\n");
+			fprintf(output, "#           time CPU                counts %*s events\n", unit_width, "unit");
 			break;
 		case AGGR_GLOBAL:
 		default:
-			fprintf(output, "#           time             counts events\n");
+			fprintf(output, "#           time             counts %*s events\n", unit_width, "unit");
 		}
 	}
 
@@ -517,6 +518,7 @@ static int __run_perf_stat(int argc, const char **argv)
 	unsigned long long t0, t1;
 	struct perf_evsel *counter;
 	struct timespec ts;
+	size_t l;
 	int status = 0;
 	const bool forks = (argc > 0);
 
@@ -566,6 +568,10 @@ static int __run_perf_stat(int argc, const char **argv)
 			return -1;
 		}
 		counter->supported = true;
+
+		l = strlen(counter->unit);
+		if (l > unit_width)
+			unit_width = l;
 	}
 
 	if (perf_evlist__apply_filters(evsel_list)) {
@@ -705,14 +711,25 @@ static void aggr_printout(struct perf_evsel *evsel, int id, int nr)
 static void nsec_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
 {
 	double msecs = avg / 1e6;
-	const char *fmt = csv_output ? "%.6f%s%s" : "%18.6f%s%-25s";
+	const char *fmt_v, *fmt_n;
 	char name[25];
 
+	fmt_v = csv_output ? "%.6f%s" : "%18.6f%s";
+	fmt_n = csv_output ? "%s" : "%-25s";
+
 	aggr_printout(evsel, cpu, nr);
 
 	scnprintf(name, sizeof(name), "%s%s",
 		  perf_evsel__name(evsel), csv_output ? "" : " (msec)");
-	fprintf(output, fmt, msecs, csv_sep, name);
+
+	fprintf(output, fmt_v, msecs, csv_sep);
+
+	if (csv_output)
+		fprintf(output, "%s%s", evsel->unit, csv_sep);
+	else
+		fprintf(output, "%-*s%s", unit_width, evsel->unit, csv_sep);
+
+	fprintf(output, fmt_n, name);
 
 	if (evsel->cgrp)
 		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
@@ -909,21 +926,31 @@ static void print_ll_cache_misses(int cpu,
 static void abs_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
 {
 	double total, ratio = 0.0, total2;
+	double sc =  evsel->scale;
 	const char *fmt;
 
-	if (csv_output)
-		fmt = "%.0f%s%s";
-	else if (big_num)
-		fmt = "%'18.0f%s%-25s";
-	else
-		fmt = "%18.0f%s%-25s";
+	if (csv_output) {
+		fmt = sc != 1.0 ?  "%.2f%s" : "%.0f%s";
+	} else {
+		if (big_num)
+			fmt = sc != 1.0 ? "%'18.2f%s" : "%'18.0f%s";
+		else
+			fmt = sc != 1.0 ? "%18.2f%s" : "%18.0f%s";
+	}
 
 	aggr_printout(evsel, cpu, nr);
 
 	if (aggr_mode == AGGR_GLOBAL)
 		cpu = 0;
 
-	fprintf(output, fmt, avg, csv_sep, perf_evsel__name(evsel));
+	fprintf(output, fmt, avg, csv_sep);
+
+	if (evsel->unit)
+		fprintf(output, "%-*s%s",
+			csv_output ? 0 : unit_width,
+			evsel->unit, csv_sep);
+
+	fprintf(output, "%-*s", csv_output ? 0 : 25, perf_evsel__name(evsel));
 
 	if (evsel->cgrp)
 		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
@@ -942,7 +969,10 @@ static void abs_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
 
 		if (total && avg) {
 			ratio = total / avg;
-			fprintf(output, "\n                                             #   %5.2f  stalled cycles per insn", ratio);
+			fprintf(output, "\n");
+			if (aggr_mode == AGGR_NONE)
+				fprintf(output, "        ");
+			fprintf(output, "                                                  #   %5.2f  stalled cycles per insn", ratio);
 		}
 
 	} else if (perf_evsel__match(evsel, HARDWARE, HW_BRANCH_MISSES) &&
@@ -1062,6 +1092,7 @@ static void print_aggr(char *prefix)
 {
 	struct perf_evsel *counter;
 	int cpu, cpu2, s, s2, id, nr;
+	double uval;
 	u64 ena, run, val;
 
 	if (!(aggr_map || aggr_get_id))
@@ -1088,11 +1119,17 @@ static void print_aggr(char *prefix)
 			if (run == 0 || ena == 0) {
 				aggr_printout(counter, id, nr);
 
-				fprintf(output, "%*s%s%*s",
+				fprintf(output, "%*s%s",
 					csv_output ? 0 : 18,
 					counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-					csv_sep,
-					csv_output ? 0 : -24,
+					csv_sep);
+
+				fprintf(output, "%-*s%s",
+					csv_output ? 0 : unit_width,
+					counter->unit, csv_sep);
+
+				fprintf(output, "%*s",
+					csv_output ? 0 : -25,
 					perf_evsel__name(counter));
 
 				if (counter->cgrp)
@@ -1102,11 +1139,12 @@ static void print_aggr(char *prefix)
 				fputc('\n', output);
 				continue;
 			}
+			uval = val * counter->scale;
 
 			if (nsec_counter(counter))
-				nsec_printout(id, nr, counter, val);
+				nsec_printout(id, nr, counter, uval);
 			else
-				abs_printout(id, nr, counter, val);
+				abs_printout(id, nr, counter, uval);
 
 			if (!csv_output) {
 				print_noise(counter, 1.0);
@@ -1129,16 +1167,21 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	struct perf_stat *ps = counter->priv;
 	double avg = avg_stats(&ps->res_stats[0]);
 	int scaled = counter->counts->scaled;
+	double uval;
 
 	if (prefix)
 		fprintf(output, "%s", prefix);
 
 	if (scaled == -1) {
-		fprintf(output, "%*s%s%*s",
+		fprintf(output, "%*s%s",
 			csv_output ? 0 : 18,
 			counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-			csv_sep,
-			csv_output ? 0 : -24,
+			csv_sep);
+		fprintf(output, "%-*s%s",
+			csv_output ? 0 : unit_width,
+			counter->unit, csv_sep);
+		fprintf(output, "%*s",
+			csv_output ? 0 : -25,
 			perf_evsel__name(counter));
 
 		if (counter->cgrp)
@@ -1148,10 +1191,12 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 		return;
 	}
 
+	uval = avg * counter->scale;
+
 	if (nsec_counter(counter))
-		nsec_printout(-1, 0, counter, avg);
+		nsec_printout(-1, 0, counter, uval);
 	else
-		abs_printout(-1, 0, counter, avg);
+		abs_printout(-1, 0, counter, uval);
 
 	print_noise(counter, avg);
 
@@ -1178,6 +1223,7 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 static void print_counter(struct perf_evsel *counter, char *prefix)
 {
 	u64 ena, run, val;
+	double uval;
 	int cpu;
 
 	for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
@@ -1189,14 +1235,20 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 			fprintf(output, "%s", prefix);
 
 		if (run == 0 || ena == 0) {
-			fprintf(output, "CPU%*d%s%*s%s%*s",
+			fprintf(output, "CPU%*d%s%*s%s",
 				csv_output ? 0 : -4,
 				perf_evsel__cpus(counter)->map[cpu], csv_sep,
 				csv_output ? 0 : 18,
 				counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-				csv_sep,
-				csv_output ? 0 : -24,
-				perf_evsel__name(counter));
+				csv_sep);
+
+				fprintf(output, "%-*s%s",
+					csv_output ? 0 : unit_width,
+					counter->unit, csv_sep);
+
+				fprintf(output, "%*s",
+					csv_output ? 0 : -25,
+					perf_evsel__name(counter));
 
 			if (counter->cgrp)
 				fprintf(output, "%s%s",
@@ -1206,10 +1258,12 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 			continue;
 		}
 
+		uval = val * counter->scale;
+
 		if (nsec_counter(counter))
-			nsec_printout(cpu, 0, counter, val);
+			nsec_printout(cpu, 0, counter, uval);
 		else
-			abs_printout(cpu, 0, counter, val);
+			abs_printout(cpu, 0, counter, uval);
 
 		if (!csv_output) {
 			print_noise(counter, 1.0);
tools/perf/util/evsel.c +2 -0
index 5280820..f1e3d02 100644
@@ -162,6 +162,8 @@ void perf_evsel__init(struct perf_evsel *evsel,
 	evsel->idx	   = idx;
 	evsel->attr	   = *attr;
 	evsel->leader	   = evsel;
+	evsel->unit	   = "";
+	evsel->scale	   = 1.0;
 	INIT_LIST_HEAD(&evsel->node);
 	hists__init(&evsel->hists);
 	evsel->sample_size = __perf_evsel__sample_size(attr->sample_type);
tools/perf/util/evsel.h +3 -0
index 64ec8e1..df2ea2b 100644
@@ -68,6 +68,8 @@ struct perf_evsel {
 	u32			ids;
 	struct hists		hists;
 	char			*name;
+	double			scale;
+	const char		*unit;
 	struct event_format	*tp_format;
 	union {
 		void		*priv;
@@ -127,6 +129,7 @@ extern const char *perf_evsel__sw_names[PERF_COUNT_SW_MAX];
 int __perf_evsel__hw_cache_type_op_res_name(u8 type, u8 op, u8 result,
 					    char *bf, size_t size);
 const char *perf_evsel__name(struct perf_evsel *evsel);
+
 const char *perf_evsel__group_name(struct perf_evsel *evsel);
 int perf_evsel__group_desc(struct perf_evsel *evsel, char *buf, size_t size);
 
tools/perf/util/parse-events.c +19 -9
index c90e55c..90ae328 100644
@@ -269,9 +269,10 @@ const char *event_type(int type)
 
 
 
-static int __add_event(struct list_head *list, int *idx,
-		       struct perf_event_attr *attr,
-		       char *name, struct cpu_map *cpus)
+static struct perf_evsel *
+__add_event(struct list_head *list, int *idx,
+	    struct perf_event_attr *attr,
+	    char *name, struct cpu_map *cpus)
 {
 	struct perf_evsel *evsel;
 
@@ -279,19 +280,19 @@ static int __add_event(struct list_head *list, int *idx,
 
 	evsel = perf_evsel__new(attr, (*idx)++);
 	if (!evsel)
-		return -ENOMEM;
+		return NULL;
 
 	evsel->cpus = cpus;
 	if (name)
 		evsel->name = strdup(name);
 	list_add_tail(&evsel->node, list);
-	return 0;
+	return evsel;
 }
 
 static int add_event(struct list_head *list, int *idx,
 		     struct perf_event_attr *attr, char *name)
 {
-	return __add_event(list, idx, attr, name, NULL);
+	return __add_event(list, idx, attr, name, NULL) ? 0 : -ENOMEM;
 }
 
 static int parse_aliases(char *str, const char *names[][PERF_EVSEL__MAX_ALIASES], int size)
@@ -633,6 +634,9 @@ int parse_events_add_pmu(struct list_head *list, int *idx,
 {
 	struct perf_event_attr attr;
 	struct perf_pmu *pmu;
+	struct perf_evsel *evsel;
+	char *unit;
+	double scale;
 
 	pmu = perf_pmu__find(name);
 	if (!pmu)
@@ -640,7 +644,7 @@ int parse_events_add_pmu(struct list_head *list, int *idx,
 
 	memset(&attr, 0, sizeof(attr));
 
-	if (perf_pmu__check_alias(pmu, head_config))
+	if (perf_pmu__check_alias(pmu, head_config, &unit, &scale))
 		return -EINVAL;
 
 	/*
@@ -652,8 +656,14 @@ int parse_events_add_pmu(struct list_head *list, int *idx,
 	if (perf_pmu__config(pmu, &attr, head_config))
 		return -EINVAL;
 
-	return __add_event(list, idx, &attr, pmu_event_name(head_config),
-			   pmu->cpus);
+	evsel = __add_event(list, idx, &attr, pmu_event_name(head_config),
+			    pmu->cpus);
+	if (evsel) {
+		evsel->unit = unit;
+		evsel->scale = scale;
+	}
+
+	return evsel ? 0 : -ENOMEM;
 }
 
 int parse_events__modifier_group(struct list_head *list,
tools/perf/util/pmu.c +134 -4

214 lines not on this page · raw message

tools/perf/util/pmu.h +2 -1
index 1179b26..9183380 100644
@@ -28,7 +28,8 @@ int perf_pmu__config(struct perf_pmu *pmu, struct perf_event_attr *attr,
 int perf_pmu__config_terms(struct list_head *formats,
 			   struct perf_event_attr *attr,
 			   struct list_head *head_terms);
-int perf_pmu__check_alias(struct perf_pmu *pmu, struct list_head *head_terms);
+int perf_pmu__check_alias(struct perf_pmu *pmu, struct list_head *head_terms,
+			  char **unit, double *scale);
 struct list_head *perf_pmu__alias(struct perf_pmu *pmu,
 				  struct list_head *head_terms);
 int perf_pmu_wrap(void);
Signature · 1 line
1.7.9.5
#4 lore raw

[tip:perf/core] tools/perf/stat: Add event unit and scale support

tools/perf/stat: Add event unit and scale support
Commit-ID:  410136f5dd96b6013fe6d1011b523b1c247e1ccb
Gitweb:     http://git.kernel.org/tip/410136f5dd96b6013fe6d1011b523b1c247e1ccb
Author:     Stephane Eranian <[email protected]>
AuthorDate: Tue, 12 Nov 2013 17:58:49 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 27 Nov 2013 11:16:39 +0100

tools/perf/stat: Add event unit and scale support

This patch adds perf stat support for handling event units and scales as exported by the kernel.

The kernel can export PMU events actual unit and scaling factor via sysfs:

  $ ls -1 /sys/devices/power/events/energy-*
  /sys/devices/power/events/energy-cores
  /sys/devices/power/events/energy-cores.scale
  /sys/devices/power/events/energy-cores.unit
  /sys/devices/power/events/energy-pkg
  /sys/devices/power/events/energy-pkg.scale
  /sys/devices/power/events/energy-pkg.unit
  $ cat /sys/devices/power/events/energy-cores.scale
  2.3283064365386962890625e-10
  $ cat cat /sys/devices/power/events/energy-cores.unit
  Joules

This patch modifies the pmu event alias code to check for the presence of the .unit and .scale files to load the corresponding values. They are then used by perf stat transparently:

   # perf stat -a -e power/energy-pkg/,power/energy-cores/,cycles -I 1000 sleep 1000
   #          time             counts   unit events
       1.000214717               3.07 Joules power/energy-pkg/         [100.00%]
       1.000214717               0.53 Joules power/energy-cores/
       1.000214717           12965028        cycles                    [100.00%]
       2.000749289               3.01 Joules power/energy-pkg/
       2.000749289               0.52 Joules power/energy-cores/
       2.000749289           15817043        cycles

When the event does not have an explicit unit exported by the kernel, nothing is printed. In csv output mode, there will be an empty field.

Special thanks to Jiri for providing the supporting code in the parser to trigger reading of the scale and unit files.

---
 tools/perf/builtin-stat.c      | 114 +++++++++++++++++++++++++---------
 tools/perf/util/evsel.c        |   2 +
 tools/perf/util/evsel.h        |   3 +
 tools/perf/util/parse-events.c |  28 ++++++---
 tools/perf/util/pmu.c          | 138 +++++++++++++++++++++++++++++++++++++++--
 tools/perf/util/pmu.h          |   3 +-
 6 files changed, 244 insertions(+), 44 deletions(-)
tools/perf/builtin-stat.c +84 -30
index ee0d565..dab98b5 100644
@@ -138,6 +138,7 @@ static const char		*post_cmd			= NULL;
 static bool			sync_run			= false;
 static unsigned int		interval			= 0;
 static unsigned int		initial_delay			= 0;
+static unsigned int		unit_width			= 4; /* strlen("unit") */
 static bool			forever				= false;
 static struct timespec		ref_time;
 static struct cpu_map		*aggr_map;
@@ -461,17 +462,17 @@ static void print_interval(void)
 	if (num_print_interval == 0 && !csv_output) {
 		switch (aggr_mode) {
 		case AGGR_SOCKET:
-			fprintf(output, "#           time socket cpus             counts events\n");
+			fprintf(output, "#           time socket cpus             counts %*s events\n", unit_width, "unit");
 			break;
 		case AGGR_CORE:
-			fprintf(output, "#           time core         cpus             counts events\n");
+			fprintf(output, "#           time core         cpus             counts %*s events\n", unit_width, "unit");
 			break;
 		case AGGR_NONE:
-			fprintf(output, "#           time CPU                 counts events\n");
+			fprintf(output, "#           time CPU                counts %*s events\n", unit_width, "unit");
 			break;
 		case AGGR_GLOBAL:
 		default:
-			fprintf(output, "#           time             counts events\n");
+			fprintf(output, "#           time             counts %*s events\n", unit_width, "unit");
 		}
 	}
 
@@ -516,6 +517,7 @@ static int __run_perf_stat(int argc, const char **argv)
 	unsigned long long t0, t1;
 	struct perf_evsel *counter;
 	struct timespec ts;
+	size_t l;
 	int status = 0;
 	const bool forks = (argc > 0);
 
@@ -565,6 +567,10 @@ static int __run_perf_stat(int argc, const char **argv)
 			return -1;
 		}
 		counter->supported = true;
+
+		l = strlen(counter->unit);
+		if (l > unit_width)
+			unit_width = l;
 	}
 
 	if (perf_evlist__apply_filters(evsel_list)) {
@@ -704,14 +710,25 @@ static void aggr_printout(struct perf_evsel *evsel, int id, int nr)
 static void nsec_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
 {
 	double msecs = avg / 1e6;
-	const char *fmt = csv_output ? "%.6f%s%s" : "%18.6f%s%-25s";
+	const char *fmt_v, *fmt_n;
 	char name[25];
 
+	fmt_v = csv_output ? "%.6f%s" : "%18.6f%s";
+	fmt_n = csv_output ? "%s" : "%-25s";
+
 	aggr_printout(evsel, cpu, nr);
 
 	scnprintf(name, sizeof(name), "%s%s",
 		  perf_evsel__name(evsel), csv_output ? "" : " (msec)");
-	fprintf(output, fmt, msecs, csv_sep, name);
+
+	fprintf(output, fmt_v, msecs, csv_sep);
+
+	if (csv_output)
+		fprintf(output, "%s%s", evsel->unit, csv_sep);
+	else
+		fprintf(output, "%-*s%s", unit_width, evsel->unit, csv_sep);
+
+	fprintf(output, fmt_n, name);
 
 	if (evsel->cgrp)
 		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
@@ -908,21 +925,31 @@ static void print_ll_cache_misses(int cpu,
 static void abs_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
 {
 	double total, ratio = 0.0, total2;
+	double sc =  evsel->scale;
 	const char *fmt;
 
-	if (csv_output)
-		fmt = "%.0f%s%s";
-	else if (big_num)
-		fmt = "%'18.0f%s%-25s";
-	else
-		fmt = "%18.0f%s%-25s";
+	if (csv_output) {
+		fmt = sc != 1.0 ?  "%.2f%s" : "%.0f%s";
+	} else {
+		if (big_num)
+			fmt = sc != 1.0 ? "%'18.2f%s" : "%'18.0f%s";
+		else
+			fmt = sc != 1.0 ? "%18.2f%s" : "%18.0f%s";
+	}
 
 	aggr_printout(evsel, cpu, nr);
 
 	if (aggr_mode == AGGR_GLOBAL)
 		cpu = 0;
 
-	fprintf(output, fmt, avg, csv_sep, perf_evsel__name(evsel));
+	fprintf(output, fmt, avg, csv_sep);
+
+	if (evsel->unit)
+		fprintf(output, "%-*s%s",
+			csv_output ? 0 : unit_width,
+			evsel->unit, csv_sep);
+
+	fprintf(output, "%-*s", csv_output ? 0 : 25, perf_evsel__name(evsel));
 
 	if (evsel->cgrp)
 		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
@@ -941,7 +968,10 @@ static void abs_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
 
 		if (total && avg) {
 			ratio = total / avg;
-			fprintf(output, "\n                                             #   %5.2f  stalled cycles per insn", ratio);
+			fprintf(output, "\n");
+			if (aggr_mode == AGGR_NONE)
+				fprintf(output, "        ");
+			fprintf(output, "                                                  #   %5.2f  stalled cycles per insn", ratio);
 		}
 
 	} else if (perf_evsel__match(evsel, HARDWARE, HW_BRANCH_MISSES) &&
@@ -1061,6 +1091,7 @@ static void print_aggr(char *prefix)
 {
 	struct perf_evsel *counter;
 	int cpu, cpu2, s, s2, id, nr;
+	double uval;
 	u64 ena, run, val;
 
 	if (!(aggr_map || aggr_get_id))
@@ -1087,11 +1118,17 @@ static void print_aggr(char *prefix)
 			if (run == 0 || ena == 0) {
 				aggr_printout(counter, id, nr);
 
-				fprintf(output, "%*s%s%*s",
+				fprintf(output, "%*s%s",
 					csv_output ? 0 : 18,
 					counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-					csv_sep,
-					csv_output ? 0 : -24,
+					csv_sep);
+
+				fprintf(output, "%-*s%s",
+					csv_output ? 0 : unit_width,
+					counter->unit, csv_sep);
+
+				fprintf(output, "%*s",
+					csv_output ? 0 : -25,
 					perf_evsel__name(counter));
 
 				if (counter->cgrp)
@@ -1101,11 +1138,12 @@ static void print_aggr(char *prefix)
 				fputc('\n', output);
 				continue;
 			}
+			uval = val * counter->scale;
 
 			if (nsec_counter(counter))
-				nsec_printout(id, nr, counter, val);
+				nsec_printout(id, nr, counter, uval);
 			else
-				abs_printout(id, nr, counter, val);
+				abs_printout(id, nr, counter, uval);
 
 			if (!csv_output) {
 				print_noise(counter, 1.0);
@@ -1128,16 +1166,21 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	struct perf_stat *ps = counter->priv;
 	double avg = avg_stats(&ps->res_stats[0]);
 	int scaled = counter->counts->scaled;
+	double uval;
 
 	if (prefix)
 		fprintf(output, "%s", prefix);
 
 	if (scaled == -1) {
-		fprintf(output, "%*s%s%*s",
+		fprintf(output, "%*s%s",
 			csv_output ? 0 : 18,
 			counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-			csv_sep,
-			csv_output ? 0 : -24,
+			csv_sep);
+		fprintf(output, "%-*s%s",
+			csv_output ? 0 : unit_width,
+			counter->unit, csv_sep);
+		fprintf(output, "%*s",
+			csv_output ? 0 : -25,
 			perf_evsel__name(counter));
 
 		if (counter->cgrp)
@@ -1147,10 +1190,12 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 		return;
 	}
 
+	uval = avg * counter->scale;
+
 	if (nsec_counter(counter))
-		nsec_printout(-1, 0, counter, avg);
+		nsec_printout(-1, 0, counter, uval);
 	else
-		abs_printout(-1, 0, counter, avg);
+		abs_printout(-1, 0, counter, uval);
 
 	print_noise(counter, avg);
 
@@ -1177,6 +1222,7 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 static void print_counter(struct perf_evsel *counter, char *prefix)
 {
 	u64 ena, run, val;
+	double uval;
 	int cpu;
 
 	for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
@@ -1188,14 +1234,20 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 			fprintf(output, "%s", prefix);
 
 		if (run == 0 || ena == 0) {
-			fprintf(output, "CPU%*d%s%*s%s%*s",
+			fprintf(output, "CPU%*d%s%*s%s",
 				csv_output ? 0 : -4,
 				perf_evsel__cpus(counter)->map[cpu], csv_sep,
 				csv_output ? 0 : 18,
 				counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-				csv_sep,
-				csv_output ? 0 : -24,
-				perf_evsel__name(counter));
+				csv_sep);
+
+				fprintf(output, "%-*s%s",
+					csv_output ? 0 : unit_width,
+					counter->unit, csv_sep);
+
+				fprintf(output, "%*s",
+					csv_output ? 0 : -25,
+					perf_evsel__name(counter));
 
 			if (counter->cgrp)
 				fprintf(output, "%s%s",
@@ -1205,10 +1257,12 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 			continue;
 		}
 
+		uval = val * counter->scale;
+
 		if (nsec_counter(counter))
-			nsec_printout(cpu, 0, counter, val);
+			nsec_printout(cpu, 0, counter, uval);
 		else
-			abs_printout(cpu, 0, counter, val);
+			abs_printout(cpu, 0, counter, uval);
 
 		if (!csv_output) {
 			print_noise(counter, 1.0);
tools/perf/util/evsel.c +2 -0
index 46dd4c2..dad6492 100644
@@ -162,6 +162,8 @@ void perf_evsel__init(struct perf_evsel *evsel,
 	evsel->idx	   = idx;
 	evsel->attr	   = *attr;
 	evsel->leader	   = evsel;
+	evsel->unit	   = "";
+	evsel->scale	   = 1.0;
 	INIT_LIST_HEAD(&evsel->node);
 	hists__init(&evsel->hists);
 	evsel->sample_size = __perf_evsel__sample_size(attr->sample_type);
tools/perf/util/evsel.h +3 -0
index 1ea7c92..8120eeb 100644
@@ -68,6 +68,8 @@ struct perf_evsel {
 	u32			ids;
 	struct hists		hists;
 	char			*name;
+	double			scale;
+	const char		*unit;
 	struct event_format	*tp_format;
 	union {
 		void		*priv;
@@ -138,6 +140,7 @@ extern const char *perf_evsel__sw_names[PERF_COUNT_SW_MAX];
 int __perf_evsel__hw_cache_type_op_res_name(u8 type, u8 op, u8 result,
 					    char *bf, size_t size);
 const char *perf_evsel__name(struct perf_evsel *evsel);
+
 const char *perf_evsel__group_name(struct perf_evsel *evsel);
 int perf_evsel__group_desc(struct perf_evsel *evsel, char *buf, size_t size);
 
tools/perf/util/parse-events.c +19 -9
index 6de6f89..969cb8f 100644
@@ -269,9 +269,10 @@ const char *event_type(int type)
 
 
 
-static int __add_event(struct list_head *list, int *idx,
-		       struct perf_event_attr *attr,
-		       char *name, struct cpu_map *cpus)
+static struct perf_evsel *
+__add_event(struct list_head *list, int *idx,
+	    struct perf_event_attr *attr,
+	    char *name, struct cpu_map *cpus)
 {
 	struct perf_evsel *evsel;
 
@@ -279,19 +280,19 @@ static int __add_event(struct list_head *list, int *idx,
 
 	evsel = perf_evsel__new_idx(attr, (*idx)++);
 	if (!evsel)
-		return -ENOMEM;
+		return NULL;
 
 	evsel->cpus = cpus;
 	if (name)
 		evsel->name = strdup(name);
 	list_add_tail(&evsel->node, list);
-	return 0;
+	return evsel;
 }
 
 static int add_event(struct list_head *list, int *idx,
 		     struct perf_event_attr *attr, char *name)
 {
-	return __add_event(list, idx, attr, name, NULL);
+	return __add_event(list, idx, attr, name, NULL) ? 0 : -ENOMEM;
 }
 
 static int parse_aliases(char *str, const char *names[][PERF_EVSEL__MAX_ALIASES], int size)
@@ -633,6 +634,9 @@ int parse_events_add_pmu(struct list_head *list, int *idx,
 {
 	struct perf_event_attr attr;
 	struct perf_pmu *pmu;
+	struct perf_evsel *evsel;
+	char *unit;
+	double scale;
 
 	pmu = perf_pmu__find(name);
 	if (!pmu)
@@ -640,7 +644,7 @@ int parse_events_add_pmu(struct list_head *list, int *idx,
 
 	memset(&attr, 0, sizeof(attr));
 
-	if (perf_pmu__check_alias(pmu, head_config))
+	if (perf_pmu__check_alias(pmu, head_config, &unit, &scale))
 		return -EINVAL;
 
 	/*
@@ -652,8 +656,14 @@ int parse_events_add_pmu(struct list_head *list, int *idx,
 	if (perf_pmu__config(pmu, &attr, head_config))
 		return -EINVAL;
 
-	return __add_event(list, idx, &attr, pmu_event_name(head_config),
-			   pmu->cpus);
+	evsel = __add_event(list, idx, &attr, pmu_event_name(head_config),
+			    pmu->cpus);
+	if (evsel) {
+		evsel->unit = unit;
+		evsel->scale = scale;
+	}
+
+	return evsel ? 0 : -ENOMEM;
 }
 
 int parse_events__modifier_group(struct list_head *list,
tools/perf/util/pmu.c +134 -4

214 lines not on this page · raw message

tools/perf/util/pmu.h +2 -1
index 1179b26..9183380 100644
@@ -28,7 +28,8 @@ int perf_pmu__config(struct perf_pmu *pmu, struct perf_event_attr *attr,
 int perf_pmu__config_terms(struct list_head *formats,
 			   struct perf_event_attr *attr,
 			   struct list_head *head_terms);
-int perf_pmu__check_alias(struct perf_pmu *pmu, struct list_head *head_terms);
+int perf_pmu__check_alias(struct perf_pmu *pmu, struct list_head *head_terms,
+			  char **unit, double *scale);
 struct list_head *perf_pmu__alias(struct perf_pmu *pmu,
 				  struct list_head *head_terms);
 int perf_pmu_wrap(void);
#5 lore raw

PATCH v7 3/4 perf,x86: add Intel RAPL PMU support

In reply to Stephane Eranian

This patch adds a new uncore PMU to expose the Intel RAPL energy consumption counters. Up to 3 counters, each counting a particular RAPL event are exposed.

The RAPL counters are available on Intel SandyBridge, IvyBridge, Haswell. The server skus add a 3rd counter.

The following events are available and exposed in sysfs:
- power/energy-cores: power consumption of all cores on socket
- power/energy-pkg: power consumption of all cores + LLc cache
- power/energy-dram: power consumption of DRAM (servers only)

For each event both the unit (Joules) and scale (2^-32 J) is exposed in sysfs for use by perf stat and other tools. Files are:
/sys/devices/power/events/energy-*.unit /sys/devices/power/events/energy-*.scale

The RAPL PMU is uncore by nature and is implemented such that it only works in system-wide mode. Measuring only one CPU per socket is sufficient. The /sys/devices/power/cpumask file can be used by tools to figure out which CPUs to monitor by default. For instance, on a 2-socket system, 2 CPUs (one on each socket) will be shown.

All the counters measure in the same unit (exposed via sysfs). The perf_events API exposes all RAPL counters as 64-bit integers counting in unit of 1/2^32 Joules (about 0.23 nJ). User level tools must convert the counts by multiplying them by 2^-32 to obtain Joules. The reason for this is that the kernel avoids doing floating point math whenever possible because it is expensive (user floating-point state must be saved). The method used avoids kernel floating-point usage. There is no loss of precision. Thanks to PeterZ for suggesting this approach.

To convert the raw count in Watt:
W = C * 2.3 / (1e10 * time)
or ldexp(C, -32).

RAPL PMU is a new standalone PMU which registers with the perf_event core subsystem. The PMU type (attr->type) is dynamically allocated and is available from /sys/device/power/type.

Sampling is not supported by the RAPL PMU. There is no privilege level filtering either.

---
 arch/x86/kernel/cpu/Makefile                |    2 +-
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |  591 +++++++++++++++++++++++++++
 2 files changed, 592 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kernel/cpu/perf_event_intel_rapl.c
arch/x86/kernel/cpu/Makefile +1 -1
index 47b56a7..6359506 100644
@@ -36,7 +36,7 @@ obj-$(CONFIG_CPU_SUP_AMD)		+= perf_event_amd_iommu.o
 endif
 obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_p6.o perf_event_knc.o perf_event_p4.o
 obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_lbr.o perf_event_intel_ds.o perf_event_intel.o
-obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_uncore.o
+obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_uncore.o perf_event_intel_rapl.o
 endif
 
 
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +591 -0

591 lines not on this page · raw message

Signature · 1 line
1.7.9.5
maintainer
#6 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Quoted 5 lines · Stephane Eranian

Sampling is not supported by the RAPL PMU. There is no privilege level filtering either.

So the merge window closed and we're starting to merge new bits for the next cycle.

Ingo fell over the double SoB here; Maria didn't actually send me this patch, so the SoB seems awkward at best.

Sob only comes from being the primary author or from collecting it in an aggregate work and passing it on.

I know people often use this double sob for co-authorship (even on single line patches), but its not supported by the rules as outlined in Documentation/SubmittingPatches.

So Ingo changed it to a Reviewed-by tag if that's fine with you - further credits should be expressed either as split-out patches that track authorship, or additional credits in the source code.

#7 lore raw

[tip:perf/core] perf/x86: Add Intel RAPL PMU support

In reply to Stephane Eranian
perf/x86: Add Intel RAPL PMU support
Commit-ID:  4788e5b4b2338f85fa42a712a182d8afd65d7c58
Gitweb:     http://git.kernel.org/tip/4788e5b4b2338f85fa42a712a182d8afd65d7c58
Author:     Stephane Eranian <[email protected]>
AuthorDate: Tue, 12 Nov 2013 17:58:50 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 27 Nov 2013 11:16:40 +0100

perf/x86: Add Intel RAPL PMU support

This patch adds a new uncore PMU to expose the Intel RAPL energy consumption counters. Up to 3 counters, each counting a particular RAPL event are exposed.

The RAPL counters are available on Intel SandyBridge, IvyBridge, Haswell. The server skus add a 3rd counter.

The following events are available and exposed in sysfs:

- power/energy-cores: power consumption of all cores on socket
- power/energy-pkg: power consumption of all cores + LLc cache
- power/energy-dram: power consumption of DRAM (servers only)

For each event both the unit (Joules) and scale (2^-32 J) is exposed in sysfs for use by perf stat and other tools. The files are:

/sys/devices/power/events/energy-*.unit
/sys/devices/power/events/energy-*.scale

The RAPL PMU is uncore by nature and is implemented such that it only works in system-wide mode. Measuring only one CPU per socket is sufficient. The /sys/devices/power/cpumask file can be used by tools to figure out which CPUs to monitor by default. For instance, on a 2-socket system, 2 CPUs (one on each socket) will be shown.

All the counters measure in the same unit (exposed via sysfs). The perf_events API exposes all RAPL counters as 64-bit integers counting in unit of 1/2^32 Joules (about 0.23 nJ). User level tools must convert the counts by multiplying them by 2^-32 to obtain Joules. The reason for this is that the kernel avoids doing floating point math whenever possible because it is expensive (user floating-point state must be saved). The method used avoids kernel floating-point usage. There is no loss of precision. Thanks to PeterZ for suggesting this approach.

To convert the raw count in Watt:
W = C * 2.3 / (1e10 * time)
or ldexp(C, -32).

RAPL PMU is a new standalone PMU which registers with the perf_event core subsystem. The PMU type (attr->type) is dynamically allocated and is available from /sys/device/power/type.

Sampling is not supported by the RAPL PMU. There is no privilege level filtering either.

---
 arch/x86/kernel/cpu/Makefile                |   2 +-
 arch/x86/kernel/cpu/perf_event_intel_rapl.c | 591 ++++++++++++++++++++++++++++
 2 files changed, 592 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/Makefile +1 -1
index 47b56a7..6359506 100644
@@ -36,7 +36,7 @@ obj-$(CONFIG_CPU_SUP_AMD)		+= perf_event_amd_iommu.o
 endif
 obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_p6.o perf_event_knc.o perf_event_p4.o
 obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_lbr.o perf_event_intel_ds.o perf_event_intel.o
-obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_uncore.o
+obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_uncore.o perf_event_intel_rapl.o
 endif
 
 
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +591 -0

591 lines not on this page · raw message

#8 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

In reply to Stephane Eranian
Quoted 6 lines · Stephane Eranian

This patch adds a new uncore PMU to expose the Intel RAPL energy consumption counters. Up to 3 counters, each counting a particular RAPL event are exposed.

The RAPL counters are available on Intel SandyBridge, IvyBridge, Haswell. The server skus add a 3rd counter.

So I notice PP1 (which is the GPU power on non-server chips) is not supported.

Is that just for simplicity?

Quoted 4 lines

The following events are available and exposed in sysfs:
- power/energy-cores: power consumption of all cores on socket
- power/energy-pkg: power consumption of all cores + LLc cache
- power/energy-dram: power consumption of DRAM (servers only)

This "power" naming seems a bit generic. If other hardware has power measurements can they be put in the same directory? power/gpu? power/usb?

Also, can support for reading the power from other vendors be put here? Like AMDs (unfortunately named) APM (active power management) power readings?

Quoted 3 lines

Files are:
/sys/devices/power/events/energy-*.unit
/sys/devices/power/events/energy-*.scale

Are all of these sys files having documentation added under Documentation/ABI?

Quoted 6 lines

The RAPL PMU is uncore by nature and is implemented such that it only works in system-wide mode. Measuring only one CPU per socket is sufficient. The /sys/devices/power/cpumask file can be used by tools to figure out which CPUs to monitor by default. For instance, on a 2-socket system, 2 CPUs (one on each socket) will be shown.

do the measurements require CAP_SYS_ADMIN like other system-wide uncore measurements? It didn't look like it in the patch but I might have missed it.

I'm sure the security people will start making claims that you can make guesses about password encryption algorithms based on the global power consumption numbers.

Sorry if these are annoying questions, I am glad to see this driver make progress, as I've had the misfortune of maintaining various user-space-MSR hacks designed to get this info because of lack of kernel support.

Vince

#9 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Quoted 6 lines · Vince Weaver
Quoted 4 lines · Stephane Eranian

This patch adds a new uncore PMU to expose the Intel RAPL energy consumption counters. Up to 3 counters, each counting a particular RAPL event are exposed.

Quoted 8 lines
Quoted 2 lines

The RAPL counters are available on Intel SandyBridge, IvyBridge, Haswell. The server skus add a 3rd counter.

So I notice PP1 (which is the GPU power on non-server chips) is not supported.

Is that just for simplicity?

Does it work on specific models only? I bet so. How to detect those?

Quoted 9 lines
Quoted 4 lines

The following events are available and exposed in sysfs:
- power/energy-cores: power consumption of all cores on socket
- power/energy-pkg: power consumption of all cores + LLc cache
- power/energy-dram: power consumption of DRAM (servers only)

This "power" naming seems a bit generic. If other hardware has power measurements can they be put in the same directory? power/gpu? power/usb?

I think so.

Quoted 11 lines

Also, can support for reading the power from other vendors be put here? Like AMDs (unfortunately named) APM (active power management) power readings?

Quoted 3 lines

Files are:
/sys/devices/power/events/energy-*.unit
/sys/devices/power/events/energy-*.scale

Are all of these sys files having documentation added under Documentation/ABI?

Not yet.

Quoted 11 lines
Quoted 6 lines

The RAPL PMU is uncore by nature and is implemented such that it only works in system-wide mode. Measuring only one CPU per socket is sufficient. The /sys/devices/power/cpumask file can be used by tools to figure out which CPUs to monitor by default. For instance, on a 2-socket system, 2 CPUs (one on each socket) will be shown.

do the measurements require CAP_SYS_ADMIN like other system-wide uncore measurements? It didn't look like it in the patch but I might have missed it.

Measurements are limited to system-wide counting. Thus, you need CAP_SYS_ADMIN.

Quoted 9 lines

I'm sure the security people will start making claims that you can make guesses about password encryption algorithms based on the global power consumption numbers.

Sorry if these are annoying questions, I am glad to see this driver make progress, as I've had the misfortune of maintaining various user-space-MSR hacks designed to get this info because of lack of kernel support.

You don't really need a driver, you can also just use modprobe msr + the turbostat utility now part of the kernel source tree. It taps into the same counters.

#10 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Quoted 8 lines · Stephane Eranian
Quoted 5 lines · Vince Weaver

So I notice PP1 (which is the GPU power on non-server chips) is not supported.

Is that just for simplicity?

Does it work on specific models only? I bet so. How to detect those?

In general it is on the machines that don't support the DRAM measurements (so the non-EP machines) but I don't know if there's a nice list anywhere.

Intel manuals say:
For a client platform, PP1 domain refers to the power plane of a specific device in the uncore. For server platforms, PP1 domain is not supported,

usually PP1 I think maps to the embedded GPU.

Quoted 7 lines
Quoted 4 lines

Sorry if these are annoying questions, I am glad to see this driver make progress, as I've had the misfortune of maintaining various user-space-MSR hacks designed to get this info because of lack of kernel support.

You don't really need a driver, you can also just use modprobe msr + the turbostat utility now part of the kernel source tree. It taps into the same counters.

yes, the code I maintain doesn't have a custom driver, it just uses the raw msr interface. That's how PAPI currently works. Though I guess I'll have to maintain both that and the new interface for a while as not everyone's going to run out and instlal 3.14.

It's a shame that it took so long for proper RAPL support to get merged. Because of the delay there are lots of people poking the MSRs themselves and various vendors who are shipping custom out-of-tree RAPL drivers, etc.

Vince

#11 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Quoted 14 lines · Vince Weaver
Quoted 8 lines · Stephane Eranian
Quoted 5 lines · Vince Weaver

So I notice PP1 (which is the GPU power on non-server chips) is not supported.

Is that just for simplicity?

Does it work on specific models only? I bet so. How to detect those?

In general it is on the machines that don't support the DRAM measurements (so the non-EP machines) but I don't know if there's a nice list anywhere.

Quoted 6 lines

Intel manuals say:
For a client platform, PP1 domain refers to the power plane of a specific device in the uncore. For server platforms, PP1 domain is not supported,

usually PP1 I think maps to the embedded GPU.

It would indeed be nice to expose PP1 too via the same facility - Haswell and later spends some 40% of the CPU die on the integrated GPU and people end up using it.

Thanks,

Ingo

#12 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Quoted 22 lines · Ingo Molnar
Quoted 19 lines · Vince Weaver
Quoted 8 lines · Stephane Eranian
Quoted 5 lines · Vince Weaver

So I notice PP1 (which is the GPU power on non-server chips) is not supported.

Is that just for simplicity?

Does it work on specific models only? I bet so. How to detect those?

In general it is on the machines that don't support the DRAM measurements (so the non-EP machines) but I don't know if there's a nice list anywhere.

Intel manuals say:
For a client platform, PP1 domain refers to the power plane of a specific device in the uncore. For server platforms, PP1 domain is not supported,

Quoted 6 lines
Quoted 1 line

usually PP1 I think maps to the embedded GPU.

It would indeed be nice to expose PP1 too via the same facility - Haswell and later spends some 40% of the CPU die on the integrated GPU and people end up using it.

My worry is to determine if the GPU is actually enabled or even present. Using the x86_model may not be enough for that.

#13 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Quoted 23 lines · Stephane Eranian
Quoted 22 lines · Ingo Molnar
Quoted 19 lines · Vince Weaver
Quoted 8 lines · Stephane Eranian
Quoted 5 lines · Vince Weaver

So I notice PP1 (which is the GPU power on non-server chips) is not supported.

Is that just for simplicity?

Does it work on specific models only? I bet so. How to detect those?

In general it is on the machines that don't support the DRAM measurements (so the non-EP machines) but I don't know if there's a nice list anywhere.

Intel manuals say:
For a client platform, PP1 domain refers to the power plane of a specific device in the uncore. For server platforms, PP1 domain is not supported,

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

usually PP1 I think maps to the embedded GPU.

It would indeed be nice to expose PP1 too via the same facility - Haswell and later spends some 40% of the CPU die on the integrated GPU and people end up using it.

My worry is to determine if the GPU is actually enabled or even present. Using the x86_model may not be enough for that.

In my experience if the device is not there you just get 0s as results from RAPL (I've also seen this on some Sandybridge-EP machines we have that for whatever reason don't support the DRAM RAPL results).

So in theory it would be harmless to export the values even if not supported. What is the worst failure mode? That somehow a recent CPU doesn't support the MSR and we get a GPF when trying to access it?

Vince

#14 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Quoted 34 lines · Vince Weaver
Quoted 23 lines · Stephane Eranian
Quoted 22 lines · Ingo Molnar
Quoted 19 lines · Vince Weaver
Quoted 8 lines · Stephane Eranian
Quoted 5 lines · Vince Weaver

So I notice PP1 (which is the GPU power on non-server chips) is not supported.

Is that just for simplicity?

Does it work on specific models only? I bet so. How to detect those?

In general it is on the machines that don't support the DRAM measurements (so the non-EP machines) but I don't know if there's a nice list anywhere.

Intel manuals say:
For a client platform, PP1 domain refers to the power plane of a specific device in the uncore. For server platforms, PP1 domain is not supported,

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

usually PP1 I think maps to the embedded GPU.

It would indeed be nice to expose PP1 too via the same facility - Haswell and later spends some 40% of the CPU die on the integrated GPU and people end up using it.

My worry is to determine if the GPU is actually enabled or even present. Using the x86_model may not be enough for that.

Quoted 8 lines

In my experience if the device is not there you just get 0s as results from RAPL (I've also seen this on some Sandybridge-EP machines we have that for whatever reason don't support the DRAM RAPL results).

So in theory it would be harmless to export the values even if not supported. What is the worst failure mode? That somehow a recent CPU doesn't support the MSR and we get a GPF when trying to access it?

Yes. I think that's what you get with DRAM on client for instance.

Quoted 1 line

Vince

maintainer
#15 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Quoted 5 lines · Stephane Eranian
Quoted 4 lines

So in theory it would be harmless to export the values even if not supported. What is the worst failure mode? That somehow a recent CPU doesn't support the MSR and we get a GPF when trying to access it?

Yes. I think that's what you get with DRAM on client for instance.

We have rdmsr_safe for non-existent MSRs, in case touching such is the issue.

Signature · 5 lines
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--
#16 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Vince,

Ok, I have added PP1 now and it works fine on my HSW desktop. What I want to know now is what is this map to on servers? And on clients, it is always mapped to the GFX. If so, then we can use a more specific name: power/energy-gfx/

I will send a patch soon.

Thanks.

Quoted 16 lines · Borislav Petkov
Quoted 5 lines · Stephane Eranian
Quoted 4 lines

So in theory it would be harmless to export the values even if not supported. What is the worst failure mode? That somehow a recent CPU doesn't support the MSR and we get a GPF when trying to access it?

Yes. I think that's what you get with DRAM on client for instance.

We have rdmsr_safe for non-existent MSRs, in case touching such is the issue.

--
Regards/Gruss,
Boris.

Sent from a fat crate under my desk. Formatting is fine. --

#17 lore raw

Re: [PATCH v7 3/4] perf,x86: add Intel RAPL PMU support

Quoted 4 lines · Stephane Eranian

Ok, I have added PP1 now and it works fine on my HSW desktop. What I want to know now is what is this map to on servers? And on clients, it is always mapped to the GFX. If so, then we can use a more specific name: power/energy-gfx/

I don't know if it is documented well, but I get the impression that PP1 is always graphics.

The turbostat code included with the kernel certainly thinks it is always graphics, it maps PP1 to "GFX"

The turbostat code also has a map of which chips have which features, although it seems to use internal Intel abbreviations that I don't necessarily follow (BYT? AVN?):

        switch (model) {
        case 0x2A:
        case 0x3A:
        case 0x3C:      /* HSW */
        case 0x3F:      /* HSW */
        case 0x45:      /* HSW */
        case 0x46:      /* HSW */
                do_rapl = RAPL_PKG | RAPL_CORES | RAPL_CORE_POLICY | 
RAPL_GFX | RAPL_PKG_POWER_INFO;
                break;
        case 0x2D:
        case 0x3E:
                do_rapl = RAPL_PKG | RAPL_CORES | RAPL_CORE_POLICY | 
RAPL_DRAM | RAPL_PKG_PERF_STATUS | RAPL_DRAM_PERF_STATUS | 
RAPL_PKG_POWER_INFO;
                break;
        case 0x37:      /* BYT */
        case 0x4D:      /* AVN */
                do_rapl = RAPL_PKG | RAPL_CORES ;
                break;
        default:
                return;
        }
#18 lore raw

PATCH v7 4/4 perf,x86: add RAPL hrtimer support

In reply to Stephane Eranian

The RAPL PMU counters do not interrupt on overflow. Therefore, the kernel needs to poll the counters to avoid missing an overflow. This patch adds the hrtimer code to do this.

The timer interval is calculated at boot time based on the power unit used by the HW.

There is one hrtimer per-cpu to handle the case of multiple simultaneous use across cores on the same package + hotplug CPU.

---
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |   74 ++++++++++++++++++++++++++-
 1 file changed, 72 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_rapl.c +72 -2
index cfcd386..a6f16ca 100644
@@ -96,6 +96,8 @@ struct rapl_pmu {
 	int		 n_active; /* number of active events */
 	struct list_head active_list;
 	struct pmu	 *pmu; /* pointer to rapl_pmu_class */
+	ktime_t		 timer_interval; /* in ktime_t unit */
+	struct hrtimer   hrtimer;
 };
 
 static struct pmu rapl_pmu_class;
@@ -158,6 +160,48 @@ static u64 rapl_event_update(struct perf_event *event)
 	return new_raw_count;
 }
 
+static void rapl_start_hrtimer(struct rapl_pmu *pmu)
+{
+	__hrtimer_start_range_ns(&pmu->hrtimer,
+			pmu->timer_interval, 0,
+			HRTIMER_MODE_REL_PINNED, 0);
+}
+
+static void rapl_stop_hrtimer(struct rapl_pmu *pmu)
+{
+	hrtimer_cancel(&pmu->hrtimer);
+}
+
+static enum hrtimer_restart rapl_hrtimer_handle(struct hrtimer *hrtimer)
+{
+	struct rapl_pmu *pmu = __get_cpu_var(rapl_pmu);
+	struct perf_event *event;
+	unsigned long flags;
+
+	if (!pmu->n_active)
+		return HRTIMER_NORESTART;
+
+	spin_lock_irqsave(&pmu->lock, flags);
+
+	list_for_each_entry(event, &pmu->active_list, active_entry) {
+		rapl_event_update(event);
+	}
+
+	spin_unlock_irqrestore(&pmu->lock, flags);
+
+	hrtimer_forward_now(hrtimer, pmu->timer_interval);
+
+	return HRTIMER_RESTART;
+}
+
+static void rapl_hrtimer_init(struct rapl_pmu *pmu)
+{
+	struct hrtimer *hr = &pmu->hrtimer;
+
+	hrtimer_init(hr, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hr->function = rapl_hrtimer_handle;
+}
+
 static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 				   struct perf_event *event)
 {
@@ -171,6 +215,8 @@ static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 	local64_set(&event->hw.prev_count, rapl_read_counter(event));
 
 	pmu->n_active++;
+	if (pmu->n_active == 1)
+		rapl_start_hrtimer(pmu);
 }
 
 static void rapl_pmu_event_start(struct perf_event *event, int mode)
@@ -195,6 +241,8 @@ static void rapl_pmu_event_stop(struct perf_event *event, int mode)
 	if (!(hwc->state & PERF_HES_STOPPED)) {
 		WARN_ON_ONCE(pmu->n_active <= 0);
 		pmu->n_active--;
+		if (pmu->n_active == 0)
+			rapl_stop_hrtimer(pmu);
 
 		list_del(&event->active_entry);
 
@@ -423,6 +471,9 @@ static void rapl_cpu_exit(int cpu)
 	 */
 	if (target >= 0)
 		perf_pmu_migrate_context(pmu->pmu, cpu, target);
+
+	/* cancel overflow polling timer for CPU */
+	rapl_stop_hrtimer(pmu);
 }
 
 static void rapl_cpu_init(int cpu)
@@ -442,6 +493,7 @@ static int rapl_cpu_prepare(int cpu)
 {
 	struct rapl_pmu *pmu = per_cpu(rapl_pmu, cpu);
 	int phys_id = topology_physical_package_id(cpu);
+	u64 ms;
 
 	if (pmu)
 		return 0;
@@ -466,6 +518,22 @@ static int rapl_cpu_prepare(int cpu)
 	pmu->hw_unit = (pmu->hw_unit >> 8) & 0x1FULL;
 	pmu->pmu = &rapl_pmu_class;
 
+	/*
+	 * use reference of 200W for scaling the timeout
+	 * to avoid missing counter overflows.
+	 * 200W = 200 Joules/sec
+	 * divide interval by 2 to avoid lockstep (2 * 100)
+	 * if hw unit is 32, then we use 2 ms 1/200/2
+	 */
+	if (pmu->hw_unit < 32)
+		ms = 1000 * (1ULL << (32 - pmu->hw_unit - 1)) / (2 * 100);
+	else
+		ms = 2;
+
+	pmu->timer_interval = ms_to_ktime(ms);
+
+	rapl_hrtimer_init(pmu);
+
 	/* set RAPL pmu for this cpu for now */
 	per_cpu(rapl_pmu, cpu) = pmu;
 	per_cpu(rapl_pmu_to_free, cpu) = NULL;
@@ -580,9 +648,11 @@ static int __init rapl_pmu_init(void)
 
 	pr_info("RAPL PMU detected, hw unit 2^-%d Joules,"
 		" API unit is 2^-32 Joules,"
-		" %d fixed counters\n",
+		" %d fixed counters"
+		" %llu ms ovfl timer\n",
 		pmu->hw_unit,
-		hweight32(rapl_cntr_mask));
+		hweight32(rapl_cntr_mask),
+		ktime_to_ms(pmu->timer_interval));
 
 	put_online_cpus();
 
Signature · 1 line
1.7.9.5
#19 lore raw

[tip:perf/core] perf/x86: Add RAPL hrtimer support

perf/x86: Add RAPL hrtimer support
Commit-ID:  eafade90e67a9b627aee331a9d0cc0c4328fb00e
Gitweb:     http://git.kernel.org/tip/eafade90e67a9b627aee331a9d0cc0c4328fb00e
Author:     Stephane Eranian <[email protected]>
AuthorDate: Tue, 12 Nov 2013 17:58:51 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 27 Nov 2013 13:45:04 +0100

perf/x86: Add RAPL hrtimer support

The RAPL PMU counters do not interrupt on overflow. Therefore, the kernel needs to poll the counters to avoid missing an overflow. This patch adds the hrtimer code to do this.

The timer interval is calculated at boot time based on the power unit used by the HW.

There is one hrtimer per-cpu to handle the case of multiple simultaneous use across cores on the same package + hotplug CPU.

---
 arch/x86/kernel/cpu/perf_event_intel_rapl.c | 74 ++++++++++++++++++++++++++++-
 1 file changed, 72 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_rapl.c +72 -2
index cfcd386..bf8e4a7 100644
@@ -96,6 +96,8 @@ struct rapl_pmu {
 	int		 n_active; /* number of active events */
 	struct list_head active_list;
 	struct pmu	 *pmu; /* pointer to rapl_pmu_class */
+	ktime_t		 timer_interval; /* in ktime_t unit */
+	struct hrtimer   hrtimer;
 };
 
 static struct pmu rapl_pmu_class;
@@ -158,6 +160,48 @@ again:
 	return new_raw_count;
 }
 
+static void rapl_start_hrtimer(struct rapl_pmu *pmu)
+{
+	__hrtimer_start_range_ns(&pmu->hrtimer,
+			pmu->timer_interval, 0,
+			HRTIMER_MODE_REL_PINNED, 0);
+}
+
+static void rapl_stop_hrtimer(struct rapl_pmu *pmu)
+{
+	hrtimer_cancel(&pmu->hrtimer);
+}
+
+static enum hrtimer_restart rapl_hrtimer_handle(struct hrtimer *hrtimer)
+{
+	struct rapl_pmu *pmu = __get_cpu_var(rapl_pmu);
+	struct perf_event *event;
+	unsigned long flags;
+
+	if (!pmu->n_active)
+		return HRTIMER_NORESTART;
+
+	spin_lock_irqsave(&pmu->lock, flags);
+
+	list_for_each_entry(event, &pmu->active_list, active_entry) {
+		rapl_event_update(event);
+	}
+
+	spin_unlock_irqrestore(&pmu->lock, flags);
+
+	hrtimer_forward_now(hrtimer, pmu->timer_interval);
+
+	return HRTIMER_RESTART;
+}
+
+static void rapl_hrtimer_init(struct rapl_pmu *pmu)
+{
+	struct hrtimer *hr = &pmu->hrtimer;
+
+	hrtimer_init(hr, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hr->function = rapl_hrtimer_handle;
+}
+
 static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 				   struct perf_event *event)
 {
@@ -171,6 +215,8 @@ static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 	local64_set(&event->hw.prev_count, rapl_read_counter(event));
 
 	pmu->n_active++;
+	if (pmu->n_active == 1)
+		rapl_start_hrtimer(pmu);
 }
 
 static void rapl_pmu_event_start(struct perf_event *event, int mode)
@@ -195,6 +241,8 @@ static void rapl_pmu_event_stop(struct perf_event *event, int mode)
 	if (!(hwc->state & PERF_HES_STOPPED)) {
 		WARN_ON_ONCE(pmu->n_active <= 0);
 		pmu->n_active--;
+		if (pmu->n_active == 0)
+			rapl_stop_hrtimer(pmu);
 
 		list_del(&event->active_entry);
 
@@ -423,6 +471,9 @@ static void rapl_cpu_exit(int cpu)
 	 */
 	if (target >= 0)
 		perf_pmu_migrate_context(pmu->pmu, cpu, target);
+
+	/* cancel overflow polling timer for CPU */
+	rapl_stop_hrtimer(pmu);
 }
 
 static void rapl_cpu_init(int cpu)
@@ -442,6 +493,7 @@ static int rapl_cpu_prepare(int cpu)
 {
 	struct rapl_pmu *pmu = per_cpu(rapl_pmu, cpu);
 	int phys_id = topology_physical_package_id(cpu);
+	u64 ms;
 
 	if (pmu)
 		return 0;
@@ -466,6 +518,22 @@ static int rapl_cpu_prepare(int cpu)
 	pmu->hw_unit = (pmu->hw_unit >> 8) & 0x1FULL;
 	pmu->pmu = &rapl_pmu_class;
 
+	/*
+	 * use reference of 200W for scaling the timeout
+	 * to avoid missing counter overflows.
+	 * 200W = 200 Joules/sec
+	 * divide interval by 2 to avoid lockstep (2 * 100)
+	 * if hw unit is 32, then we use 2 ms 1/200/2
+	 */
+	if (pmu->hw_unit < 32)
+		ms = (1000 / (2 * 100)) * (1ULL << (32 - pmu->hw_unit - 1));
+	else
+		ms = 2;
+
+	pmu->timer_interval = ms_to_ktime(ms);
+
+	rapl_hrtimer_init(pmu);
+
 	/* set RAPL pmu for this cpu for now */
 	per_cpu(rapl_pmu, cpu) = pmu;
 	per_cpu(rapl_pmu_to_free, cpu) = NULL;
@@ -580,9 +648,11 @@ static int __init rapl_pmu_init(void)
 
 	pr_info("RAPL PMU detected, hw unit 2^-%d Joules,"
 		" API unit is 2^-32 Joules,"
-		" %d fixed counters\n",
+		" %d fixed counters"
+		" %llu ms ovfl timer\n",
 		pmu->hw_unit,
-		hweight32(rapl_cntr_mask));
+		hweight32(rapl_cntr_mask),
+		ktime_to_ms(pmu->timer_interval));
 
 	put_online_cpus();
 
#20 lore raw

[tip:perf/core] perf/x86: Add RAPL hrtimer support

In reply to Stephane Eranian
perf/x86: Add RAPL hrtimer support
Commit-ID:  65661f96d3b32f4b28fef26d21be81d7e173b965
Gitweb:     http://git.kernel.org/tip/65661f96d3b32f4b28fef26d21be81d7e173b965
Author:     Stephane Eranian <[email protected]>
AuthorDate: Tue, 12 Nov 2013 17:58:51 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 27 Nov 2013 15:31:23 +0100

perf/x86: Add RAPL hrtimer support

The RAPL PMU counters do not interrupt on overflow. Therefore, the kernel needs to poll the counters to avoid missing an overflow. This patch adds the hrtimer code to do this.

The timer interval is calculated at boot time based on the power unit used by the HW.

There is one hrtimer per-cpu to handle the case of multiple simultaneous use across cores on the same package + hotplug CPU.

Thanks to Maria Dimakopoulou for her contributions to this patch especially on the math aspects.

---
 arch/x86/kernel/cpu/perf_event_intel_rapl.c | 74 ++++++++++++++++++++++++++++-
 1 file changed, 72 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_rapl.c +72 -2
index cfcd386..bf8e4a7 100644
@@ -96,6 +96,8 @@ struct rapl_pmu {
 	int		 n_active; /* number of active events */
 	struct list_head active_list;
 	struct pmu	 *pmu; /* pointer to rapl_pmu_class */
+	ktime_t		 timer_interval; /* in ktime_t unit */
+	struct hrtimer   hrtimer;
 };
 
 static struct pmu rapl_pmu_class;
@@ -158,6 +160,48 @@ again:
 	return new_raw_count;
 }
 
+static void rapl_start_hrtimer(struct rapl_pmu *pmu)
+{
+	__hrtimer_start_range_ns(&pmu->hrtimer,
+			pmu->timer_interval, 0,
+			HRTIMER_MODE_REL_PINNED, 0);
+}
+
+static void rapl_stop_hrtimer(struct rapl_pmu *pmu)
+{
+	hrtimer_cancel(&pmu->hrtimer);
+}
+
+static enum hrtimer_restart rapl_hrtimer_handle(struct hrtimer *hrtimer)
+{
+	struct rapl_pmu *pmu = __get_cpu_var(rapl_pmu);
+	struct perf_event *event;
+	unsigned long flags;
+
+	if (!pmu->n_active)
+		return HRTIMER_NORESTART;
+
+	spin_lock_irqsave(&pmu->lock, flags);
+
+	list_for_each_entry(event, &pmu->active_list, active_entry) {
+		rapl_event_update(event);
+	}
+
+	spin_unlock_irqrestore(&pmu->lock, flags);
+
+	hrtimer_forward_now(hrtimer, pmu->timer_interval);
+
+	return HRTIMER_RESTART;
+}
+
+static void rapl_hrtimer_init(struct rapl_pmu *pmu)
+{
+	struct hrtimer *hr = &pmu->hrtimer;
+
+	hrtimer_init(hr, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hr->function = rapl_hrtimer_handle;
+}
+
 static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 				   struct perf_event *event)
 {
@@ -171,6 +215,8 @@ static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 	local64_set(&event->hw.prev_count, rapl_read_counter(event));
 
 	pmu->n_active++;
+	if (pmu->n_active == 1)
+		rapl_start_hrtimer(pmu);
 }
 
 static void rapl_pmu_event_start(struct perf_event *event, int mode)
@@ -195,6 +241,8 @@ static void rapl_pmu_event_stop(struct perf_event *event, int mode)
 	if (!(hwc->state & PERF_HES_STOPPED)) {
 		WARN_ON_ONCE(pmu->n_active <= 0);
 		pmu->n_active--;
+		if (pmu->n_active == 0)
+			rapl_stop_hrtimer(pmu);
 
 		list_del(&event->active_entry);
 
@@ -423,6 +471,9 @@ static void rapl_cpu_exit(int cpu)
 	 */
 	if (target >= 0)
 		perf_pmu_migrate_context(pmu->pmu, cpu, target);
+
+	/* cancel overflow polling timer for CPU */
+	rapl_stop_hrtimer(pmu);
 }
 
 static void rapl_cpu_init(int cpu)
@@ -442,6 +493,7 @@ static int rapl_cpu_prepare(int cpu)
 {
 	struct rapl_pmu *pmu = per_cpu(rapl_pmu, cpu);
 	int phys_id = topology_physical_package_id(cpu);
+	u64 ms;
 
 	if (pmu)
 		return 0;
@@ -466,6 +518,22 @@ static int rapl_cpu_prepare(int cpu)
 	pmu->hw_unit = (pmu->hw_unit >> 8) & 0x1FULL;
 	pmu->pmu = &rapl_pmu_class;
 
+	/*
+	 * use reference of 200W for scaling the timeout
+	 * to avoid missing counter overflows.
+	 * 200W = 200 Joules/sec
+	 * divide interval by 2 to avoid lockstep (2 * 100)
+	 * if hw unit is 32, then we use 2 ms 1/200/2
+	 */
+	if (pmu->hw_unit < 32)
+		ms = (1000 / (2 * 100)) * (1ULL << (32 - pmu->hw_unit - 1));
+	else
+		ms = 2;
+
+	pmu->timer_interval = ms_to_ktime(ms);
+
+	rapl_hrtimer_init(pmu);
+
 	/* set RAPL pmu for this cpu for now */
 	per_cpu(rapl_pmu, cpu) = pmu;
 	per_cpu(rapl_pmu_to_free, cpu) = NULL;
@@ -580,9 +648,11 @@ static int __init rapl_pmu_init(void)
 
 	pr_info("RAPL PMU detected, hw unit 2^-%d Joules,"
 		" API unit is 2^-32 Joules,"
-		" %d fixed counters\n",
+		" %d fixed counters"
+		" %llu ms ovfl timer\n",
 		pmu->hw_unit,
-		hweight32(rapl_cntr_mask));
+		hweight32(rapl_cntr_mask),
+		ktime_to_ms(pmu->timer_interval));
 
 	put_online_cpus();