Skip to contentSkip to conversation

perf,x86: add Intel RAPL PMU support

PATCH Superseded Stephane Eranian 6 participants 21 messages Oct 10, 2013 – Oct 23, 2013

Subsystems (2)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover perf,x86: add Intel RAPL PMU support

This patch adds a new uncore PMU to expose the Intel RAPL energy consumption counters. Up to 3 counters, each counting a particular RAPL event are exposed.

The RAPL counters are available on Intel SandyBridge, IvyBridge, Haswell. The server skus add a 3rd counter.

The following events are available nd exposed in sysfs:
- rapl-energy-cores: power consumption of all cores on socket
- rapl-energy-pkg: power consumption of all cores + LLc cache
- rapl-energy-dram: power consumption of DRAM

The RAPL PMU is uncore by nature and is implemented such that it only works in system-wide mode. Measuring only one CPU per socket is sufficient. The /sys/devices/rapl/cpumask is exported and can be used by tools to figure out which CPU to monitor by default. For instance, on a 2-socket system, 2 CPUs (one on each socket) will be shown.

The counters all count in the same unit. The perf_events API exposes all RAPL counters as 64-bit integers counting in unit of 1/2^32 Joules (or 0.23 nJ). User level tools must convert the counts by multiplying them by 0.23 and divide 10^9 to obtain Joules. The reason for this is that the kernel avoids doing floating point math whenever possible because it is expensive (user floating-point state must be saved). The method used avoids kernel floating-point and minimizes the loss of precision (bits). Thanks to PeterZ for suggesting this approach.

To convert the raw count in Watt: W = C * 0.23 / (1e9 * time)

RAPL PMU is a new standalone PMU which registers with the perf_event core subsystem. The PMU type (attr->type) is dynamically allocated and is available from /sys/device/rapl/type.

Sampling is not supported by the RAPL PMU. There is no privilege level filtering either.

The PMU exports a cpumask in /sys/devices/uncore/cpumask. It is used by perf to ensure only one instance of each RAPL event is measured per processor socket. Hotplug CPU is also supported.

The second patch adds a hrtimer to poll the counters given that they do no interrupt on overflow. Hardware counters are 32-bit wide.

In v2, we add the locking necesarry to protect the rapl_pmu struct. We also add a description at the top of the file. We check for Intel only processor. We improved the data layout of the rapl_pmu struct. We also lifted the restriction of the number of instances of RAPL counters that can be active at the same time. RAPL is free running counters, so ought to be able to measure events as many times as necessary in parallel via multiple tools. There is never multiplexing among RAPL events.

Supported CPUs: SandyBridge, IvyBridge, Haswell.

$ perf stat -a -e rapl/rapl-energy-cores/,rapl/rapl-energy-pkg/ -I 1000 sleep 10
            time             counts events
     1.000345931        772 278 493 rapl/rapl-energy-cores/
     1.000345931     55 539 138 560 rapl/rapl-energy-pkg/    
     2.000836387        771 751 936 rapl/rapl-energy-cores/  
     2.000836387     55 326 015 488 rapl/rapl-energy-pkg/

Stephane Eranian (3):
perf: add active_entry list head to struct perf_event perf,x86: add Intel RAPL PMU support perf,x86: add RAPL hrtimer support

 arch/x86/kernel/cpu/Makefile                |    2 +-
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |  688 +++++++++++++++++++++++++++
 include/linux/perf_event.h                  |    1 +
 kernel/events/core.c                        |    1 +
 4 files changed, 691 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kernel/cpu/perf_event_intel_rapl.c
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH v2 1/3 perf: add active_entry list head to struct perf_event

This patch adds a new fields to the struct perf_event. It is intended to be used to chain events which are active (enabled). It helps in the hardware layer for PMU which do not have actual counter restrictions, i.e., free running read-only counters. Active events are chained as opposed to being tracked via the counter they use.

---
 include/linux/perf_event.h |    1 +
 kernel/events/core.c       |    1 +
 2 files changed, 2 insertions(+)
include/linux/perf_event.h +1 -0
index 2e069d1..a376384 100644
@@ -435,6 +435,7 @@ struct perf_event {
 	struct perf_cgroup		*cgrp; /* cgroup event is attach to */
 	int				cgrp_defer_enabled;
 #endif
+	struct list_head		active_entry;
 
 #endif /* CONFIG_PERF_EVENTS */
 };
kernel/events/core.c +1 -0
index c716385..b1dbf79 100644
@@ -6629,6 +6629,7 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	INIT_LIST_HEAD(&event->event_entry);
 	INIT_LIST_HEAD(&event->sibling_list);
 	INIT_LIST_HEAD(&event->rb_entry);
+	INIT_LIST_HEAD(&event->active_entry);
 
 	init_waitqueue_head(&event->waitq);
 	init_irq_work(&event->pending, perf_pending_event);
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH v2 2/3 perf,x86: add Intel RAPL PMU support

In reply to Stephane Eranian

This patch adds a new uncore PMU to expose the Intel RAPL energy consumption counters. Up to 3 counters, each counting a particular RAPL event are exposed.

The RAPL counters are available on Intel SandyBridge, IvyBridge, Haswell. The server skus add a 3rd counter.

The following events are available nd exposed in sysfs:
- rapl-energy-cores: power consumption of all cores on socket
- rapl-energy-pkg: power consumption of all cores + LLc cache
- rapl-energy-dram: power consumption of DRAM

The RAPL PMU is uncore by nature and is implemented such that it only works in system-wide mode. Measuring only one CPU per socket is sufficient. The /sys/devices/rapl/cpumask is exported and can be used by tools to figure out which CPU to monitor by default. For instance, on a 2-socket system, 2 CPUs (one on each socket) will be shown.

The counters all count in the same unit. The perf_events API exposes all RAPL counters as 64-bit integers counting in unit of 1/2^32 Joules (or 0.23 nJ). User level tools must convert the counts by multiplying them by 0.23 and divide 10^9 to obtain Joules. The reason for this is that the kernel avoids doing floating point math whenever possible because it is expensive (user floating-point state must be saved). The method used avoids kernel floating-point and minimizes the loss of precision (bits). Thanks to PeterZ for suggesting this approach.

To convert the raw count in Watt:
W = C * 0.23 / (1e9 * time)
or ldexp(C, -32).

RAPL PMU is a new standalone PMU which registers with the perf_event core subsystem. The PMU type (attr->type) is dynamically allocated and is available from /sys/device/rapl/type.

Sampling is not supported by the RAPL PMU. There is no privilege level filtering either.

---
 arch/x86/kernel/cpu/Makefile                |    2 +-
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |  623 +++++++++++++++++++++++++++
 2 files changed, 624 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kernel/cpu/perf_event_intel_rapl.c
arch/x86/kernel/cpu/Makefile +1 -1
index 47b56a7..6359506 100644
@@ -36,7 +36,7 @@ obj-$(CONFIG_CPU_SUP_AMD)		+= perf_event_amd_iommu.o
 endif
 obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_p6.o perf_event_knc.o perf_event_p4.o
 obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_lbr.o perf_event_intel_ds.o perf_event_intel.o
-obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_uncore.o
+obj-$(CONFIG_CPU_SUP_INTEL)		+= perf_event_intel_uncore.o perf_event_intel_rapl.o
 endif
 
 
arch/x86/kernel/cpu/perf_event_intel_rapl.c new file +623 -0

623 lines not on this page · raw message

Signature · 1 line
1.7.9.5
#3 lore raw

PATCH v2 3/3 perf,x86: add RAPL hrtimer support

In reply to Stephane Eranian

The RAPL PMU counters do not interrupt on overflow. Therefore, the kernel needs to poll the counters to avoid missing an overflow. This patch adds the hrtimer code to do this.

The timer internval is calculated at boot time based on the power unit used by the HW.

---
 arch/x86/kernel/cpu/perf_event_intel_rapl.c |   75 +++++++++++++++++++++++++--
 1 file changed, 70 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_rapl.c +70 -5
index abaaf4f..c5a6f51 100644
@@ -92,11 +92,13 @@ static struct kobj_attribute format_attr_##_var =		\
 
 struct rapl_pmu {
 	spinlock_t	lock;
-	atomic_t	refcnt;
 	int		hw_unit;  /* 1/2^hw_unit Joule */
-	int		phys_id;
-	int		n_active; /* number of active events */
+	struct hrtimer	hrtimer;
 	struct list_head active_list;
+	ktime_t		timer_interval; /* in ktime_t unit */
+	int		n_active; /* number of active events */
+	int		phys_id;
+	atomic_t	refcnt;
 };
 
 static struct pmu rapl_pmu_class;
@@ -161,6 +163,47 @@ static u64 rapl_event_update(struct perf_event *event)
 	return new_raw_count;
 }
 
+static void rapl_start_hrtimer(struct rapl_pmu *pmu)
+{
+	__hrtimer_start_range_ns(&pmu->hrtimer,
+			pmu->timer_interval, 0,
+			HRTIMER_MODE_REL_PINNED, 0);
+}
+
+static void rapl_stop_hrtimer(struct rapl_pmu *pmu)
+{
+	hrtimer_cancel(&pmu->hrtimer);
+}
+
+static enum hrtimer_restart rapl_hrtimer_handle(struct hrtimer *hrtimer)
+{
+	struct rapl_pmu *pmu = container_of(hrtimer, struct rapl_pmu, hrtimer);
+	struct perf_event *event;
+	unsigned long flags;
+
+	if (!pmu->n_active)
+		return HRTIMER_NORESTART;
+
+	spin_lock_irqsave(&pmu->lock, flags);
+
+	list_for_each_entry(event, &pmu->active_list, active_entry) {
+		rapl_event_update(event);
+	}
+
+	spin_unlock_irqrestore(&pmu->lock, flags);
+
+	hrtimer_forward_now(&pmu->hrtimer, pmu->timer_interval);
+
+	return HRTIMER_RESTART;
+}
+
+static void rapl_hrtimer_init(struct rapl_pmu *pmu)
+{
+	hrtimer_init(&pmu->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	pmu->hrtimer.function = rapl_hrtimer_handle;
+}
+
+
 static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 				   struct perf_event *event)
 {
@@ -174,6 +217,8 @@ static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
 	local64_set(&event->hw.prev_count, rapl_read_counter(event));
 
 	pmu->n_active++;
+	if (pmu->n_active == 1)
+		rapl_start_hrtimer(pmu);
 }
 
 static void rapl_pmu_event_start(struct perf_event *event, int mode)
@@ -198,6 +243,8 @@ static void rapl_pmu_event_stop(struct perf_event *event, int mode)
 	if (!(hwc->state & PERF_HES_STOPPED)) {
 		WARN_ON_ONCE(pmu->n_active <= 0);
 		pmu->n_active--;
+		if (pmu->n_active == 0)
+			rapl_stop_hrtimer(pmu);
 
 		list_del(&event->active_entry);
 
@@ -416,6 +463,7 @@ static int rapl_cpu_prepare(int cpu)
 {
 	struct rapl_pmu *pmu = per_cpu(rapl_pmu, cpu);
 	int phys_id = topology_physical_package_id(cpu);
+	u64 ms;
 
 	if (pmu)
 		return 0;
@@ -441,6 +489,20 @@ static int rapl_cpu_prepare(int cpu)
 	rdmsrl(MSR_RAPL_POWER_UNIT, pmu->hw_unit);
 	pmu->hw_unit = (pmu->hw_unit >> 8) & 0x1FULL;
 
+	/*
+	 * use reference of 200W for scaling the timeout
+	 * to avoid missing counter overflows.
+	 * 200W = 200 Joules/sec
+	 * divide interval by 2 to avoid lockstep (2 * 100)
+	 * if hw unit is 32, then we use 2 ms 1/200/2
+	 */
+	if (pmu->hw_unit < 32)
+		ms = 1000 * (1ULL << (32 - pmu->hw_unit - 1)) / (2 * 100);
+	else
+		ms = 2;
+
+	pmu->timer_interval = ms_to_ktime(ms);
+
 	/* set RAPL pmu for this cpu for now */
 	per_cpu(rapl_pmu_kfree, cpu) = NULL;
 	per_cpu(rapl_pmu, cpu) = pmu;
@@ -602,6 +664,7 @@ static int __init rapl_pmu_init(void)
 		}
 		rapl_cpu_prepare(cpu);
 		cpumask_set_cpu(cpu, &rapl_cpu_mask);
+		rapl_hrtimer_init(per_cpu(rapl_pmu, cpu));
 	}
 
 	perf_cpu_notifier(rapl_cpu_notifier);
@@ -612,9 +675,11 @@ static int __init rapl_pmu_init(void)
 	pmu = __get_cpu_var(rapl_pmu);
 	pr_info("RAPL PMU detected, hw unit 2^-%d Joules,"
 		" API unit is 2^-32 Joules,"
-		" %d fixed counters\n",
+		" %d fixed counters"
+		" %llu ms ovfl timer\n",
 		pmu->hw_unit,
-		hweight32(rapl_cntr_mask));
+		hweight32(rapl_cntr_mask),
+		ktime_to_ms(pmu->timer_interval));
 
 	put_online_cpus();
 
Signature · 1 line
1.7.9.5
maintainer
#5 lore raw
In reply to Stephane Eranian
Quoted 6 lines · Stephane Eranian
$ perf stat -a -e rapl/rapl-energy-cores/,rapl/rapl-energy-pkg/ -I 1000 sleep 10
            time             counts events
     1.000345931        772 278 493 rapl/rapl-energy-cores/
     1.000345931     55 539 138 560 rapl/rapl-energy-pkg/    
     2.000836387        771 751 936 rapl/rapl-energy-cores/  
     2.000836387     55 326 015 488 rapl/rapl-energy-pkg/

Hmm, so I'm looking at builtin-stat.c::print_interval() and since it gets the perf_evsel counters and you can deduce the counter name from it, you probably could match the rapl counters and do the Watts conversion above as a special case.

I dunno, it is much better than having some naked numbers for which people have to go stare at the sources + CPU vendor docs as to what they actually mean.

Thanks.

Signature · 5 lines
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--
#6 lore raw

So, the RAPL patch-set clearly needs more work.

Quoted 7 lines · Borislav Petkov
Quoted 6 lines · Stephane Eranian
$ perf stat -a -e rapl/rapl-energy-cores/,rapl/rapl-energy-pkg/ -I 1000 sleep 10
            time             counts events
     1.000345931        772 278 493 rapl/rapl-energy-cores/
     1.000345931     55 539 138 560 rapl/rapl-energy-pkg/    
     2.000836387        771 751 936 rapl/rapl-energy-cores/  
     2.000836387     55 326 015 488 rapl/rapl-energy-pkg/

Why is there the rapl/rapl duplication in the event name? It should be rapl/energy-cores, rapl/energy-pkg, etc.

I'm also not sure about the Intel-specific naming. Joules per core and Joules per socket ought to be pretty generic, even if the initial implementation is Intel-only. I.e.:

power/energy-core
power/energy-pkg

Quoted 8 lines

Hmm, so I'm looking at builtin-stat.c::print_interval() and since it gets the perf_evsel counters and you can deduce the counter name from it, you probably could match the rapl counters and do the Watts conversion above as a special case.

I dunno, it is much better than having some naked numbers for which people have to go stare at the sources + CPU vendor docs as to what they actually mean.

So what should happen here is to extend the sysfs attributes that tell us that it's in 32.32 fixed-point format.

We should also tell user-space that the unit of this counter is 'Joule'.

Then things like:

perf stat -a -e power/* sleep 1

would output, without knowing any RAPL details:

            0.20619 Joule    power/energy-core
            2.42151 Joule    power/energy-pkg

or so.

Other platforms offering energy measurement facilities will then name their counters in the same power/* (or energy/*) namespace, with new names if they do something fundamentally differently.

Tooling can then generalize along these abstractions, as much as the hardware allows it.

Thanks,

Ingo

#7 lore raw
Quoted 13 lines · Ingo Molnar

So, the RAPL patch-set clearly needs more work.

Quoted 7 lines · Borislav Petkov
Quoted 6 lines · Stephane Eranian
$ perf stat -a -e rapl/rapl-energy-cores/,rapl/rapl-energy-pkg/ -I 1000 sleep 10
            time             counts events
     1.000345931        772 278 493 rapl/rapl-energy-cores/
     1.000345931     55 539 138 560 rapl/rapl-energy-pkg/
     2.000836387        771 751 936 rapl/rapl-energy-cores/
     2.000836387     55 326 015 488 rapl/rapl-energy-pkg/
Quoted 3 lines

Why is there the rapl/rapl duplication in the event name? It should be rapl/energy-cores, rapl/energy-pkg, etc.

yeah, I thought about doing that too. I will change the names.

Quoted 4 lines

I'm also not sure about the Intel-specific naming. Joules per core and Joules per socket ought to be pretty generic, even if the initial implementation is Intel-only. I.e.:

Joules per cores (with an s)
Joules per package.
Joules per dram, i.e., all the DRAM attached to a socket (I think).

Quoted 3 lines

power/energy-core
power/energy-pkg

Fine with me. Or joules-cores to make the unit explicit

Quoted 5 lines
Quoted 5 lines

Hmm, so I'm looking at builtin-stat.c::print_interval() and since it gets the perf_evsel counters and you can deduce the counter name from it, you probably could match the rapl counters and do the Watts conversion above as a special case.

Quoted 7 lines
Quoted 3 lines

I dunno, it is much better than having some naked numbers for which people have to go stare at the sources + CPU vendor docs as to what they actually mean.

So what should happen here is to extend the sysfs attributes that tell us that it's in 32.32 fixed-point format.

We could add that in sysfs, but then I am wondering how would the tool realize it has to use this file. We'd have to create something generic like a scaling factor. If the file is there, then use it, if not assume 1x. Is that what you are thinking about?

Quoted 11 lines

We should also tell user-space that the unit of this counter is 'Joule'.

Then things like:

perf stat -a -e power/* sleep 1

would output, without knowing any RAPL details:

            0.20619 Joule    power/energy-core
            2.42151 Joule    power/energy-pkg

Not sure there is already some support for this in perf stat. Arnaldo? If not that we need another sysfs file to export the unit. Another possibility is for perf stat to recognize the power/* and extract the unit from the event name. In my example power/joules-cores -> joules.

#8 lore raw

Em Wed, Oct 16, 2013 at 03:13:54PM +0200, Stephane Eranian escreveu:

Quoted 5 lines · Ingo Molnar
Quoted 4 lines

We should also tell user-space that the unit of this counter is 'Joule'.

Then things like:

Quoted 8 lines
Quoted 7 lines

perf stat -a -e power/* sleep 1

would output, without knowing any RAPL details:

            0.20619 Joule    power/energy-core
            2.42151 Joule    power/energy-pkg

Not sure there is already some support for this in perf stat. Arnaldo?

Nope, there is not, we would have to have some table somewhere with "event-regexp: unit-string"

Quoted 3 lines

If not that we need another sysfs file to export the unit. Another possibility is for perf stat to recognize the power/* and extract the unit from the event name. In my example power/joules-cores -> joules.

I.e. you would be encoding the counter unit as the suffix, might as well call it "power/cores.joules" and use the dot as the separator for the unit, but would be just a compact form to encode the counter->unit table.

- Arnaldo

#9 lore raw
Quoted 18 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 16, 2013 at 03:13:54PM +0200, Stephane Eranian escreveu:

Quoted 5 lines · Ingo Molnar
Quoted 4 lines

We should also tell user-space that the unit of this counter is 'Joule'.

Then things like:

Quoted 8 lines
Quoted 7 lines

perf stat -a -e power/* sleep 1

would output, without knowing any RAPL details:

            0.20619 Joule    power/energy-core
            2.42151 Joule    power/energy-pkg

Not sure there is already some support for this in perf stat. Arnaldo?

Nope, there is not, we would have to have some table somewhere with "event-regexp: unit-string"

Quoted 9 lines
Quoted 3 lines

If not that we need another sysfs file to export the unit. Another possibility is for perf stat to recognize the power/* and extract the unit from the event name. In my example power/joules-cores -> joules.

I.e. you would be encoding the counter unit as the suffix, might as well call it "power/cores.joules" and use the dot as the separator for the unit, but would be just a compact form to encode the counter->unit table.

May be easier to add a sysfs entry with the unit to display.

#10 lore raw
Quoted 24 lines · Stephane Eranian
Quoted 22 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 16, 2013 at 03:13:54PM +0200, Stephane Eranian escreveu:

Quoted 5 lines · Ingo Molnar
Quoted 4 lines

We should also tell user-space that the unit of this counter is 'Joule'.

Then things like:

Quoted 8 lines
Quoted 7 lines

perf stat -a -e power/* sleep 1

would output, without knowing any RAPL details:

            0.20619 Joule    power/energy-core
            2.42151 Joule    power/energy-pkg

Not sure there is already some support for this in perf stat. Arnaldo?

Nope, there is not, we would have to have some table somewhere with "event-regexp: unit-string"

Quoted 3 lines

If not that we need another sysfs file to export the unit. Another possibility is for perf stat to recognize the power/* and extract the unit from the event name. In my example power/joules-cores -> joules.

Quoted 6 lines
Quoted 4 lines

I.e. you would be encoding the counter unit as the suffix, might as well call it "power/cores.joules" and use the dot as the separator for the unit, but would be just a compact form to encode the counter->unit table.

May be easier to add a sysfs entry with the unit to display.

Yes - with no entry meaning a raw 'count' or such.

Thanks,

Ingo

maintainer
#11 lore raw
Quoted 8 lines · Ingo Molnar
Quoted 6 lines
Quoted 4 lines

I.e. you would be encoding the counter unit as the suffix, might as well call it "power/cores.joules" and use the dot as the separator for the unit, but would be just a compact form to encode the counter->unit table.

May be easier to add a sysfs entry with the unit to display.

Yes - with no entry meaning a raw 'count' or such.

The downside to such a sysfs entry will be the scope. It would either be pmu wide (unwieldy for many PMUs) or be only per listed event; and we really don't want exhaustive event lists in the kernel.

maintainer
#12 lore raw
Quoted 3 lines · Peter Zijlstra

The downside to such a sysfs entry will be the scope. It would either be pmu wide (unwieldy for many PMUs) or be only per listed event; and we really don't want exhaustive event lists in the kernel.

So why not teach perf tool to recognize the PMU instead of adding anything to the kernel?

It seems much easier to me...

Signature · 5 lines
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--
#13 lore raw
In reply to Peter Zijlstra

Peter,

Quoted 6 lines · Peter Zijlstra
Quoted 5 lines · Ingo Molnar
Quoted 5 lines
Quoted 4 lines

I.e. you would be encoding the counter unit as the suffix, might as well call it "power/cores.joules" and use the dot as the separator for the unit, but would be just a compact form to encode the counter->unit table.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

May be easier to add a sysfs entry with the unit to display.

Yes - with no entry meaning a raw 'count' or such.

The downside to such a sysfs entry will be the scope. It would either be pmu wide (unwieldy for many PMUs) or be only per listed event; and we really don't want exhaustive event lists in the kernel.

Why not put in the events subdir:

/sys/devices/power/events/energy-cores
/sys/devices/power/events/energy-cores.unit
/sys/devices/power/events/energy-cores.scaling

$ cat energy-core.unit
Joules
$ cat energy-core.scaling
0.00000000023

Perf could easily lookup those files and if they are not present it will print the event as it does today. If present, then it will print the unit and apply the scaling factor to the raw cont (already scaled for multiplexing).

Borislav, the scaling factor cannot be hardcoded into perf because it can change for processor to processor.

#14 lore raw
In reply to Ingo Molnar

Hi,

I have updated my RAPL patches to implement the suggested changes. I will post the patch very soon. The new look and feel is as folllows:

# perf stat -a -e power/energy-cores/,power/energy-pkg/,ref-cycles -I
1000 sleep 1000
#           time        unit            counts events
     1.000264953      Joules              2.09 power/energy-cores/
  [100.00%]
     1.000264953      Joules              5.94 power/energy-pkg/
     1.000264953                   160,530,320 ref-cycles
     2.000640422      Joules              2.07 power/energy-cores/
     2.000640422      Joules              5.94 power/energy-pkg/
     2.000640422                   152,673,056 ref-cycles
     3.000964416      Joules              2.08 power/energy-cores/
     3.000964416      Joules              5.93 power/energy-pkg/
     3.000964416                   158,779,184 ref-cycles
# ls -1 /sys/devices/power/events/
energy-cores
energy-cores.scale
energy-cores.unit
energy-pkg
energy-pkg.scale
energy-pkg.unit
# cat /sys/devices/power/events/energy-cores.scale
2.3e-10
# cat /sys/devices/power/events/energy-cores.unit
Joules

Of course, this unit and scaling support is generic and not limited to the RAPL events. For now, this only works with events exported by the kernel via sysfs.

Quoted 39 lines · Ingo Molnar
Quoted 24 lines · Stephane Eranian
Quoted 22 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 16, 2013 at 03:13:54PM +0200, Stephane Eranian escreveu:

Quoted 5 lines · Ingo Molnar
Quoted 4 lines

We should also tell user-space that the unit of this counter is 'Joule'.

Then things like:

Quoted 8 lines
Quoted 7 lines

perf stat -a -e power/* sleep 1

would output, without knowing any RAPL details:

            0.20619 Joule    power/energy-core
            2.42151 Joule    power/energy-pkg

Not sure there is already some support for this in perf stat. Arnaldo?

Nope, there is not, we would have to have some table somewhere with "event-regexp: unit-string"

Quoted 3 lines

If not that we need another sysfs file to export the unit. Another possibility is for perf stat to recognize the power/* and extract the unit from the event name. In my example power/joules-cores -> joules.

Quoted 6 lines
Quoted 4 lines

I.e. you would be encoding the counter unit as the suffix, might as well call it "power/cores.joules" and use the dot as the separator for the unit, but would be just a compact form to encode the counter->unit table.

May be easier to add a sysfs entry with the unit to display.

Yes - with no entry meaning a raw 'count' or such.

Thanks,

Ingo

#15 lore raw

Em Tue, Oct 22, 2013 at 06:47:38PM +0200, Stephane Eranian escreveu:

Quoted 16 lines

I have updated my RAPL patches to implement the suggested changes. I will post the patch very soon. The new look and feel is as folllows:

# perf stat -a -e power/energy-cores/,power/energy-pkg/,ref-cycles -I
1000 sleep 1000
#           time        unit            counts events
     1.000264953      Joules              2.09 power/energy-cores/
  [100.00%]
     1.000264953      Joules              5.94 power/energy-pkg/
     1.000264953                   160,530,320 ref-cycles
     2.000640422      Joules              2.07 power/energy-cores/
     2.000640422      Joules              5.94 power/energy-pkg/
     2.000640422                   152,673,056 ref-cycles
     3.000964416      Joules              2.08 power/energy-cores/
     3.000964416      Joules              5.93 power/energy-pkg/
     3.000964416                   158,779,184 ref-cycles

What about:

 # perf stat -a -e power/energy-cores/,power/energy-pkg/,ref-cycles -I 1000 sleep 1000
 #           time                     events
      1.000264953              2.09 Joules power/energy-cores/
      1.000264953              5.94 Joules power/energy-pkg/
      1.000264953       160,530,320 ref-cycles
      2.000640422              2.07 Joules power/energy-cores/
      2.000640422              5.94 Joules power/energy-pkg/
      2.000640422       152,673,056 ref-cycles
      3.000964416              2.08 Joules power/energy-cores/
      3.000964416              5.93 Joules power/energy-pkg/
      3.000964416       158,779,184 ref-cycles

?

Or even 2.09J power/energy-cores/?

I.e. a perf_evsel__fprintf_value(evsel) would append a unit string, if available.

- Arnaldo

Quoted 60 lines
# ls -1 /sys/devices/power/events/
energy-cores
energy-cores.scale
energy-cores.unit
energy-pkg
energy-pkg.scale
energy-pkg.unit
# cat /sys/devices/power/events/energy-cores.scale
2.3e-10
# cat /sys/devices/power/events/energy-cores.unit
Joules

Of course, this unit and scaling support is generic and not limited to the RAPL events. For now, this only works with events exported by the kernel via sysfs.

Quoted 39 lines · Ingo Molnar
Quoted 24 lines · Stephane Eranian
Quoted 22 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 16, 2013 at 03:13:54PM +0200, Stephane Eranian escreveu:

Quoted 5 lines · Ingo Molnar
Quoted 4 lines

We should also tell user-space that the unit of this counter is 'Joule'.

Then things like:

Quoted 8 lines
Quoted 7 lines

perf stat -a -e power/* sleep 1

would output, without knowing any RAPL details:

            0.20619 Joule    power/energy-core
            2.42151 Joule    power/energy-pkg

Not sure there is already some support for this in perf stat. Arnaldo?

Nope, there is not, we would have to have some table somewhere with "event-regexp: unit-string"

Quoted 3 lines

If not that we need another sysfs file to export the unit. Another possibility is for perf stat to recognize the power/* and extract the unit from the event name. In my example power/joules-cores -> joules.

Quoted 6 lines
Quoted 4 lines

I.e. you would be encoding the counter unit as the suffix, might as well call it "power/cores.joules" and use the dot as the separator for the unit, but would be just a compact form to encode the counter->unit table.

May be easier to add a sysfs entry with the unit to display.

Yes - with no entry meaning a raw 'count' or such.

Thanks,

Ingo

#16 lore raw

Arnaldo,

Quoted 32 lines · Arnaldo Carvalho de Melo

Em Tue, Oct 22, 2013 at 06:47:38PM +0200, Stephane Eranian escreveu:

Quoted 16 lines

I have updated my RAPL patches to implement the suggested changes. I will post the patch very soon. The new look and feel is as folllows:

# perf stat -a -e power/energy-cores/,power/energy-pkg/,ref-cycles -I
1000 sleep 1000
#           time        unit            counts events
     1.000264953      Joules              2.09 power/energy-cores/
  [100.00%]
     1.000264953      Joules              5.94 power/energy-pkg/
     1.000264953                   160,530,320 ref-cycles
     2.000640422      Joules              2.07 power/energy-cores/
     2.000640422      Joules              5.94 power/energy-pkg/
     2.000640422                   152,673,056 ref-cycles
     3.000964416      Joules              2.08 power/energy-cores/
     3.000964416      Joules              5.93 power/energy-pkg/
     3.000964416                   158,779,184 ref-cycles

What about:

 # perf stat -a -e power/energy-cores/,power/energy-pkg/,ref-cycles -I 1000 sleep 1000
 #           time                     events
      1.000264953              2.09 Joules power/energy-cores/
      1.000264953              5.94 Joules power/energy-pkg/
      1.000264953       160,530,320 ref-cycles
      2.000640422              2.07 Joules power/energy-cores/
      2.000640422              5.94 Joules power/energy-pkg/
      2.000640422       152,673,056 ref-cycles
      3.000964416              2.08 Joules power/energy-cores/
      3.000964416              5.93 Joules power/energy-pkg/
      3.000964416       158,779,184 ref-cycles
Quoted 4 lines

?

Or even 2.09J power/energy-cores/?

I can try that.

Quoted 3 lines

I.e. a perf_evsel__fprintf_value(evsel) would append a unit string, if available.

I don't have this function is my tree yet (tip.git).

Quoted 63 lines

- Arnaldo

Quoted 60 lines
# ls -1 /sys/devices/power/events/
energy-cores
energy-cores.scale
energy-cores.unit
energy-pkg
energy-pkg.scale
energy-pkg.unit
# cat /sys/devices/power/events/energy-cores.scale
2.3e-10
# cat /sys/devices/power/events/energy-cores.unit
Joules

Of course, this unit and scaling support is generic and not limited to the RAPL events. For now, this only works with events exported by the kernel via sysfs.

Quoted 39 lines · Ingo Molnar
Quoted 24 lines · Stephane Eranian
Quoted 22 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 16, 2013 at 03:13:54PM +0200, Stephane Eranian escreveu:

Quoted 5 lines · Ingo Molnar
Quoted 4 lines

We should also tell user-space that the unit of this counter is 'Joule'.

Then things like:

Quoted 8 lines
Quoted 7 lines

perf stat -a -e power/* sleep 1

would output, without knowing any RAPL details:

            0.20619 Joule    power/energy-core
            2.42151 Joule    power/energy-pkg

Not sure there is already some support for this in perf stat. Arnaldo?

Nope, there is not, we would have to have some table somewhere with "event-regexp: unit-string"

Quoted 3 lines

If not that we need another sysfs file to export the unit. Another possibility is for perf stat to recognize the power/* and extract the unit from the event name. In my example power/joules-cores -> joules.

Quoted 6 lines
Quoted 4 lines

I.e. you would be encoding the counter unit as the suffix, might as well call it "power/cores.joules" and use the dot as the separator for the unit, but would be just a compact form to encode the counter->unit table.

May be easier to add a sysfs entry with the unit to display.

Yes - with no entry meaning a raw 'count' or such.

Thanks,

Ingo

#17 lore raw

Em Wed, Oct 23, 2013 at 11:34:42AM +0200, Stephane Eranian escreveu:

Quoted 18 lines

On Wed, Oct 23, 2013 at 12:18 AM, Arnaldo Carvalho de Melo

Quoted 14 lines

What about:

 # perf stat -a -e power/energy-cores/,power/energy-pkg/,ref-cycles -I 1000 sleep 1000
 #           time                     events
      1.000264953              2.09 Joules power/energy-cores/
      1.000264953              5.94 Joules power/energy-pkg/
      1.000264953       160,530,320 ref-cycles
      2.000640422              2.07 Joules power/energy-cores/
      2.000640422              5.94 Joules power/energy-pkg/
      2.000640422       152,673,056 ref-cycles
      3.000964416              2.08 Joules power/energy-cores/
      3.000964416              5.93 Joules power/energy-pkg/
      3.000964416       158,779,184 ref-cycles
Quoted 2 lines

?
Or even 2.09J power/energy-cores/?

Quoted 6 lines

I can try that.

Quoted 2 lines

I.e. a perf_evsel__fprintf_value(evsel) would append a unit string, if available.

I don't have this function is my tree yet (tip.git).

That would be a new one :-)

At some point I'll study the %pM, etc things in the kernel printk code to come up with something like perf_evsel__{f,scn}printf that allows us to use just one string format and then pick things like units as a modifier, but till then having these fprintf variants seems good enough.

- Arnaldo

#18 lore raw

Arnaldo,

Quoted 24 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 23, 2013 at 11:34:42AM +0200, Stephane Eranian escreveu:

Quoted 22 lines

On Wed, Oct 23, 2013 at 12:18 AM, Arnaldo Carvalho de Melo

Quoted 14 lines

What about:

 # perf stat -a -e power/energy-cores/,power/energy-pkg/,ref-cycles -I 1000 sleep 1000
 #           time                     events
      1.000264953              2.09 Joules power/energy-cores/
      1.000264953              5.94 Joules power/energy-pkg/
      1.000264953       160,530,320 ref-cycles
      2.000640422              2.07 Joules power/energy-cores/
      2.000640422              5.94 Joules power/energy-pkg/
      2.000640422       152,673,056 ref-cycles
      3.000964416              2.08 Joules power/energy-cores/
      3.000964416              5.93 Joules power/energy-pkg/
      3.000964416       158,779,184 ref-cycles
Quoted 2 lines

?
Or even 2.09J power/energy-cores/?

I can try that.

Quoted 2 lines

I.e. a perf_evsel__fprintf_value(evsel) would append a unit string, if available.

Quoted 9 lines
Quoted 1 line

I don't have this function is my tree yet (tip.git).

That would be a new one :-)

At some point I'll study the %pM, etc things in the kernel printk code to come up with something like perf_evsel__{f,scn}printf that allows us to use just one string format and then pick things like units as a modifier, but till then having these fprintf variants seems good enough.

Having the printf() would only be good to print the value but the problem is that you'd need to synchronize with the column headers and width. So if you say fprintf_value() print the count + unit, then you need to line up also with the column header which comes from somwhere else. I am talking about the interval printing mode here.

#19 lore raw
In reply to Stephane Eranian
Quoted 5 lines
# perf stat -a -e power/energy-cores/,power/energy-pkg/,ref-cycles -I
1000 sleep 1000
#           time        unit            counts events
     1.000264953      Joules              2.09 power/energy-cores/
  [100.00%]
Quoted 8 lines
     1.000264953      Joules              5.94 power/energy-pkg/
     1.000264953                   160,530,320 ref-cycles
     2.000640422      Joules              2.07 power/energy-cores/
     2.000640422      Joules              5.94 power/energy-pkg/
     2.000640422                   152,673,056 ref-cycles
     3.000964416      Joules              2.08 power/energy-cores/
     3.000964416      Joules              5.93 power/energy-pkg/
     3.000964416                   158,779,184 ref-cycles

Can you add some column marker that there is no unit (like -) ?

This is just in case someone wants to parse this with a tool. Yes they should be using -x, but it is still better to be always parseable.

-Andi

#20 lore raw

Andi,

Quoted 14 lines · Andi Kleen
Quoted 5 lines
# perf stat -a -e power/energy-cores/,power/energy-pkg/,ref-cycles -I
1000 sleep 1000
#           time        unit            counts events
     1.000264953      Joules              2.09 power/energy-cores/
  [100.00%]
Quoted 8 lines
     1.000264953      Joules              5.94 power/energy-pkg/
     1.000264953                   160,530,320 ref-cycles
     2.000640422      Joules              2.07 power/energy-cores/
     2.000640422      Joules              5.94 power/energy-pkg/
     2.000640422                   152,673,056 ref-cycles
     3.000964416      Joules              2.08 power/energy-cores/
     3.000964416      Joules              5.93 power/energy-pkg/
     3.000964416                   158,779,184 ref-cycles
Quoted 5 lines

Can you add some column marker that there is no unit (like -) ?

This is just in case someone wants to parse this with a tool. Yes they should be using -x, but it is still better to be always parseable.

It is parseable, it's just that you get an empty field: ,, But I can add a "?".