perf/x86: Add Intel power cstate PMUs support
Subsystems (2)
PATCH v2 1/1 perf/x86: Add Intel power cstate PMUs support
From: Kan Liang <[email protected]>
This patch adds new PMUs to support power cstate related free running (read-only) counters. These counters may be used simultaneously by other tools, such as turbostat. However, it still make sense to implement them in perf. Because we can conveniently collect them together with other events, and allow to use them from tools without special MSR access code.
These counters include CORE_C*_RESIDENCY and PKG_C*_RESIDENCY. According to counters' scope and category, two PMUs are registered with the perf_event core subsystem.
- 'cstate_core': The counter is available for each processor core. The counters include CORE_C*_RESIDENCY.
- 'cstate_pkg': The counter is available for each physical package. The counters include PKG_C*_RESIDENCY.
The events are exposed in sysfs for use by perf stat and other tools. The files are:
/sys/devices/cstate_core/events/c*-residency /sys/devices/cstate_pkg/events/c*-residency
These events only support system-wide mode counting.
The /sys/devices/cstate_*/cpumask file can be used by tools to figure out which CPUs to monitor by default.
The PMU type (attr->type) is dynamically allocated and is available from /sys/devices/core_misc/type and /sys/device/cstate_*/type.
Sampling is not supported.
Here is an example.
- To caculate the fraction of time when the core is running in C6 state CORE_C6_time% = CORE_C6_RESIDENCY / TSC
$ perf stat -x, -e"cstate_core/c6-residency/,msr/tsc/" -C0 -- taskset -c 0 sleep 5 11453299029,,cstate_core/c6-residency/,4998723500,100.00 11476922553,,msr/tsc/,4998738042,100.00 For sleep, 99.8% of time run in C6 state.
$ perf stat -x, -e"cstate_core/c6-residency/,msr/tsc/" -C0 -- taskset -c 0 busyloop 1253316,,cstate_core/c6-residency/,4360969154,100.00 10012635248,,msr/tsc/,4360972366,100.00 For busyloop, 0.01% of time run in C6 state.
---
Changes since V1
- Kernel take care of mapping pkg c6 to correct msr
arch/x86/kernel/cpu/Makefile | 1 + arch/x86/kernel/cpu/perf_event_intel_cstate.c | 750 ++++++++++++++++++++++++++ 2 files changed, 751 insertions(+) create mode 100644 arch/x86/kernel/cpu/perf_event_intel_cstate.c
arch/x86/kernel/cpu/Makefile +1 -0
@@ -41,6 +41,7 @@ obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_p6.o perf_event_knc.o perf_event_p4.o obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_intel_lbr.o perf_event_intel_ds.o perf_event_intel.o obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_intel_rapl.o perf_event_intel_cqm.o obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_intel_pt.o perf_event_intel_bts.o +obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_intel_cstate.o obj-$(CONFIG_PERF_EVENTS_INTEL_UNCORE) += perf_event_intel_uncore.o \ perf_event_intel_uncore_snb.o \
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +750 -0
750 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Peter Zijlstra
maintainerOn Mon, Jul 27, 2015 at 04:46:16AM -0400, [email protected] wrote:
As a general comment; this thing is unreadable. Far too much macro foo to instantiate the different PMUs.
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 111
Quoted 13 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +13
-0+struct perf_power_cstate_event_msr { +int id; +u64 msr; +}; + +enum perf_power_cstate_id { +/* +* power_cstate events, generalized by the kernel: +*/ +PERF_POWER_CORE_C1_RES =0 , +PERF_POWER_CORE_C3_RES , +PERF_POWER_CORE_C6_RES , +PERF_POWER_CORE_C7_RES ,
These are two different PMUs, why are they in the same enum space?
Quoted 23 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +23
-0+PERF_POWER_PKG_C2_RES , +PERF_POWER_PKG_C3_RES , +PERF_POWER_PKG_C6_RES , +PERF_POWER_PKG_C7_RES , +PERF_POWER_PKG_C8_RES , +PERF_POWER_PKG_C9_RES , +PERF_POWER_PKG_C10_RES , + + PERF_POWER_CSTATE_EVENT_MAX, /* non-ABI */ +}; + +struct power_cstate_pmu { +struct intel_power_cstate_type *power_cstate_type; +struct pmu *pmu; +}; + + +static struct intel_power_cstate_type *empty_power_cstate[] = {NULL , }; +struct intel_power_cstate_type **power_cstate = empty_power_cstate; + +static struct perf_power_cstate_event_msr power_cstate_events[] = { + {PERF_POWER_CORE_C1_RES ,MSR_CORE_C1_RES }, + { PERF_POWER_CORE_C3_RES, MSR_CORE_C3_RESIDENCY },
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 144
Quoted 10 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +10
-0+ { PERF_POWER_CORE_C6_RES, MSR_CORE_C6_RESIDENCY }, + { PERF_POWER_CORE_C7_RES, MSR_CORE_C7_RESIDENCY }, + { PERF_POWER_PKG_C2_RES, MSR_PKG_C2_RESIDENCY }, + { PERF_POWER_PKG_C3_RES, MSR_PKG_C3_RESIDENCY }, + { PERF_POWER_PKG_C6_RES, MSR_PKG_C6_RESIDENCY }, + { PERF_POWER_PKG_C7_RES, MSR_PKG_C7_RESIDENCY }, + { PERF_POWER_PKG_C8_RES, MSR_PKG_C8_RESIDENCY }, + { PERF_POWER_PKG_C9_RES, MSR_PKG_C9_RESIDENCY }, + { PERF_POWER_PKG_C10_RES, MSR_PKG_C10_RESIDENCY }, +};
What's the point of the first entry? You already know which index it is, no point in storing that again.
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 150
Quoted 7 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +6
-0+ +EVENT_ATTR_STR (c1-residency, power_core_c1_res,"event=0x00" ); +EVENT_ATTR_STR (c3-residency, power_core_c3_res,"event=0x01" ); +EVENT_ATTR_STR (c6-residency, power_core_c6_res,"event=0x02" ); +EVENT_ATTR_STR (c7-residency, power_core_c7_res,"event=0x03" ); +EVENT_ATTR_STR (c2-residency, power_pkg_c2_res,"event=0x04" );
I would very much expect this thing to start counting at 0 again.
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 158
Quoted 8 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +8
-0+EVENT_ATTR_STR (c3-residency, power_pkg_c3_res,"event=0x05" ); +EVENT_ATTR_STR (c6-residency, power_pkg_c6_res,"event=0x06" ); +EVENT_ATTR_STR (c7-residency, power_pkg_c7_res,"event=0x07" ); +EVENT_ATTR_STR (c8-residency, power_pkg_c8_res,"event=0x08" ); +EVENT_ATTR_STR (c9-residency, power_pkg_c9_res,"event=0x09" ); +EVENT_ATTR_STR (c10-residency, power_pkg_c10_res,"event=0x0a" ); + +static cpumask_t power_cstate_core_cpu_mask;
That one typically does not need a cpumask.
Quoted 15 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +14
-0+static cpumask_t power_cstate_pkg_cpu_mask; +static int power_cstate_pmu_event_init (struct perf_event *event) +{ +u64 cfg = event->attr.config; +int ret =0 ; + +if (event->attr.type != event->pmu->type) +return -ENOENT ; + +/* +* check event is known (determines counter) +*/ +if (cfg >=PERF_POWER_CSTATE_EVENT_MAX ) +return -EINVAL ;
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 188
Quoted 10 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +10
-0+ +/* unsupported modes and filters */ +if (event->attr.exclude_user || + event->attr.exclude_kernel || + event->attr.exclude_hv || + event->attr.exclude_idle || + event->attr.exclude_host || + event->attr.exclude_guest || + event->attr.sample_period)/* no sampling */ +return -EINVAL ;
Same as with the MSR thing, but worse. What happens if we hand a config value for the 'wrong' PMU type?
What happens if we hand a pkg c10 config to one that doesn't have c10?
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 202
Quoted 13 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +13
-0+/* must be done before validate_group */ + event->hw.event_base = power_cstate_events[cfg].msr; + event->hw.config = cfg; + event->hw.idx =-1 ; + +return ret; +} + +static inline u64 power_cstate_pmu_read_counter (struct perf_event *event) +{ +u64 val; + +rdmsrl_safe (event->hw.event_base, &val);
No, at this point you had better know if that MSR exists or not. If it does not the event should not exist.
Quoted 2 lines
(quoted diff) +2
-0+ return val; +}
Quoted 58 lines · Peter Zijlstra
On Mon, Jul 27, 2015 at 04:46:16AM -0400, [email protected] wrote:
As a general comment; this thing is unreadable. Far too much macro foo to instantiate the different PMUs.
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 111
Quoted 13 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +13
-0+struct perf_power_cstate_event_msr { +int id; +u64 msr; +}; + +enum perf_power_cstate_id { +/* +* power_cstate events, generalized by the kernel: +*/ +PERF_POWER_CORE_C1_RES =0 , +PERF_POWER_CORE_C3_RES , +PERF_POWER_CORE_C6_RES , +PERF_POWER_CORE_C7_RES ,These are two different PMUs, why are they in the same enum space?
Quoted 23 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +23
-0+PERF_POWER_PKG_C2_RES , +PERF_POWER_PKG_C3_RES , +PERF_POWER_PKG_C6_RES , +PERF_POWER_PKG_C7_RES , +PERF_POWER_PKG_C8_RES , +PERF_POWER_PKG_C9_RES , +PERF_POWER_PKG_C10_RES , + + PERF_POWER_CSTATE_EVENT_MAX, /* non-ABI */ +}; + +struct power_cstate_pmu { +struct intel_power_cstate_type *power_cstate_type; +struct pmu *pmu; +}; + + +static struct intel_power_cstate_type *empty_power_cstate[] = {NULL , }; +struct intel_power_cstate_type **power_cstate = empty_power_cstate; + +static struct perf_power_cstate_event_msr power_cstate_events[] = { + {PERF_POWER_CORE_C1_RES ,MSR_CORE_C1_RES }, + { PERF_POWER_CORE_C3_RES, MSR_CORE_C3_RESIDENCY },arch/x86/kernel/cpu/perf_event_intel_cstate.c line 144
Quoted 10 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +10
-0+ { PERF_POWER_CORE_C6_RES, MSR_CORE_C6_RESIDENCY }, + { PERF_POWER_CORE_C7_RES, MSR_CORE_C7_RESIDENCY }, + { PERF_POWER_PKG_C2_RES, MSR_PKG_C2_RESIDENCY }, + { PERF_POWER_PKG_C3_RES, MSR_PKG_C3_RESIDENCY }, + { PERF_POWER_PKG_C6_RES, MSR_PKG_C6_RESIDENCY }, + { PERF_POWER_PKG_C7_RES, MSR_PKG_C7_RESIDENCY }, + { PERF_POWER_PKG_C8_RES, MSR_PKG_C8_RESIDENCY }, + { PERF_POWER_PKG_C9_RES, MSR_PKG_C9_RESIDENCY }, + { PERF_POWER_PKG_C10_RES, MSR_PKG_C10_RESIDENCY }, +};What's the point of the first entry? You already know which index it is, no point in storing that again.
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ +EVENT_ATTR_STR(c1-residency, power_core_c1_res, "event=0x00");
I would not use an event code of 0. Start at 0x1
Quoted 8 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 150
Quoted 5 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +4
-0+EVENT_ATTR_STR (c3-residency, power_core_c3_res,"event=0x01" ); +EVENT_ATTR_STR (c6-residency, power_core_c6_res,"event=0x02" ); +EVENT_ATTR_STR (c7-residency, power_core_c7_res,"event=0x03" ); +EVENT_ATTR_STR (c2-residency, power_pkg_c2_res,"event=0x04" );I would very much expect this thing to start counting at 0 again.
Quoted 11 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 158
Quoted 8 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +8
-0+EVENT_ATTR_STR (c3-residency, power_pkg_c3_res,"event=0x05" ); +EVENT_ATTR_STR (c6-residency, power_pkg_c6_res,"event=0x06" ); +EVENT_ATTR_STR (c7-residency, power_pkg_c7_res,"event=0x07" ); +EVENT_ATTR_STR (c8-residency, power_pkg_c8_res,"event=0x08" ); +EVENT_ATTR_STR (c9-residency, power_pkg_c9_res,"event=0x09" ); +EVENT_ATTR_STR (c10-residency, power_pkg_c10_res,"event=0x0a" ); + +static cpumask_t power_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my understanding.
Quoted 20 lines
Quoted 20 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +19
-0+static cpumask_t power_cstate_pkg_cpu_mask; +static int power_cstate_pmu_event_init (struct perf_event *event) +{ +u64 cfg = event->attr.config; +int ret =0 ; + +if (event->attr.type != event->pmu->type) +return -ENOENT ; + +/* +* check event is known (determines counter) +*/ +if (cfg >=PERF_POWER_CSTATE_EVENT_MAX ) +return -EINVAL ; + +/* unsupported modes and filters */ + if (event->attr.exclude_user || + event->attr.exclude_kernel || + event->attr.exclude_hv ||
Quoted 9 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 188
Quoted 5 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +5
-0+ event->attr.exclude_idle || + event->attr.exclude_host || + event->attr.exclude_guest || + event->attr.sample_period) /* no sampling */ + return -EINVAL;Same as with the MSR thing, but worse. What happens if we hand a config value for the 'wrong' PMU type?
I think that's taken care of by the first test in the function.
Quoted 23 lines
What happens if we hand a pkg c10 config to one that doesn't have c10?
arch/x86/kernel/cpu/perf_event_intel_cstate.c line 202
Quoted 13 lines
arch/x86/kernel/cpu/perf_event_intel_cstate.c new file +13
-0+/* must be done before validate_group */ + event->hw.event_base = power_cstate_events[cfg].msr; + event->hw.config = cfg; + event->hw.idx =-1 ; + +return ret; +} + +static inline u64 power_cstate_pmu_read_counter (struct perf_event *event) +{ +u64 val; + +rdmsrl_safe (event->hw.event_base, &val);No, at this point you had better know if that MSR exists or not. If it does not the event should not exist.
Quoted 2 lines
(quoted diff) +2
-0+ return val; +}
I understand that these metrics are useful and needed however if I look at the broader
picture I see many PMUs doing similar things or appearing different when they are actually
very close. It would be nice to have a more unified approach. You have RAPL (client, server)
which appears as the power PMU. You have the PCU uncore on servers which also provides
C-state residency info. Yet, all these appear differently and expose events with different names.
I think we could benefit from a more unifie approach here such that you would be able to do
$ perf stat -a -e power/c6-residency/, power/energy-pkg/
on client and server without having to change the pmu name of the event names.
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my understanding.
CORE_C*_RESIDENCY are available for physical processor core.
So logical processor in same physical processor core share the same counter.
I think we need the cpumask to identify the default logical processor which do counting.
Quoted 11 lines
Quoted 1 line
I understand that these metrics are useful and needed however if I look at the broader picture I see many PMUs doing similar things or appearing different when they are actually very close. It would be nice to have a more unified approach. You have RAPL (client, server) which appears as the power PMU. You have the PCU uncore on servers which also provides C-state residency info. Yet, all these appear differently and expose events with different names.
I think we could benefit from a more unifie approach here such that you would be able to do
Quoted 4 lines
$ perf stat -a -e power/c6-residency/, power/energy-pkg/on client and server without having to change the pmu name of the event names.
Yes, I agree. I'll think about it.
Thanks,
Kan ���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 10 lines · Kan
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my understanding.
Quoted 6 lines
CORE_C*_RESIDENCY are available for physical processor core.
So logical processor in same physical processor core share the same counter.
I think we need the cpumask to identify the default logical processor which do counting.
Did you restrict these events to system-wide mode only?
Quoted 20 lines
Quoted 11 lines
Quoted 1 line
I understand that these metrics are useful and needed however if I look at the broader picture I see many PMUs doing similar things or appearing different when they are actually very close. It would be nice to have a more unified approach. You have RAPL (client, server) which appears as the power PMU. You have the PCU uncore on servers which also provides C-state residency info. Yet, all these appear differently and expose events with different names.
I think we could benefit from a more unifie approach here such that you would be able to doQuoted 4 lines
$ perf stat -a -e power/c6-residency/, power/energy-pkg/on client and server without having to change the pmu name of the event names.
Yes, I agree. I'll think about it.
Thanks,
Kan
Quoted 13 lines · Kan
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my understanding.
Quoted 8 lines
Quoted 6 lines
CORE_C*_RESIDENCY are available for physical processor core. So logical processor in same physical processor core share the same counter.
I think we need the cpumask to identify the default logical processor which do counting.Did you restrict these events to system-wide mode only?
Yes ���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 13 lines · Kan
Quoted 13 lines · Kan
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my understanding.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
CORE_C*_RESIDENCY are available for physical processor core. So logical processor in same physical processor core share the same counter.
I think we need the cpumask to identify the default logical processor which do counting.Did you restrict these events to system-wide mode only?
Ok, so that means that your cpumask includes one HT per physical core. But then, the result is not the simple aggregation of all the N/2 CPUs.
Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my
understanding.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
CORE_C*_RESIDENCY are available for physical processor core. So logical processor in same physical processor core share the same counter.
I think we need the cpumask to identify the default logical processor which do counting.
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Did you restrict these events to system-wide mode only?
Ok, so that means that your cpumask includes one HT per physical core. But then, the result is not the simple aggregation of all the N/2 CPUs.
The counter counts per physical core. The result is the aggregation of
all HT cpus in same physical core.
It's similar as per socket counter. But the scope is physical core now. ���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 16 lines · Kan
Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my
understanding.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
CORE_C*_RESIDENCY are available for physical processor core. So logical processor in same physical processor core share the same counter.
I think we need the cpumask to identify the default logical processor which do counting.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Did you restrict these events to system-wide mode only?
Ok, so that means that your cpumask includes one HT per physical core. But then, the result is not the simple aggregation of all the N/2 CPUs.
The counter counts per physical core. The result is the aggregation of all HT cpus in same physical core.
But then don't you need to divide by 2 to get a meaningful result?
Quoted 30 lines
-----Original Message-----
From: Stephane Eranian [mailto:[email protected]]
Sent: Thursday, August 06, 2015 4:38 PM
To: Liang, Kan
Cc: Peter Zijlstra; [email protected]; Arnaldo Carvalho de Melo; [email protected]; LKML
Subject: Re: [PATCH V2 1/1] perf/x86: Add Intel power cstate PMUs supportQuoted 18 lines · Kan
Quoted 18 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my
understanding.
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines
Quoted 1 line
CORE_C*_RESIDENCY are available for physical processor core. So logical processor in same physical processor core share the same counter.
I think we need the cpumask to identify the default logical processor which do counting.Did you restrict these events to system-wide mode only?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Ok, so that means that your cpumask includes one HT per physical core. But then, the result is not the simple aggregation of all the N/2 CPUs.
The counter counts per physical core. The result is the aggregation of all HT cpus in same physical core.
But then don't you need to divide by 2 to get a meaningful result?
Rethink of it. I think I was unclear about the aggregation of all HT cpus in same physical core.
physical core Cstate should equal to min(logical core C-state). So only all logical core enters C6-state, the physical core enters C6-state, then CORE_C6_RESIDENCY counts.
So if we only count on one logical core/HT for CORE_C6_RESIDENCY. We don't need to divide by 2. The count result is the residency when all logical core in C6 (some may deeper).
Thanks,
Kan
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 41 lines · Kan
Quoted 30 lines
-----Original Message-----
From: Stephane Eranian [mailto:[email protected]]
Sent: Thursday, August 06, 2015 4:38 PM
To: Liang, Kan
Cc: Peter Zijlstra; [email protected]; Arnaldo Carvalho de Melo; [email protected]; LKML
Subject: Re: [PATCH V2 1/1] perf/x86: Add Intel power cstate PMUs supportQuoted 18 lines · Kan
Quoted 18 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my
understanding.
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines
Quoted 1 line
CORE_C*_RESIDENCY are available for physical processor core. So logical processor in same physical processor core share the same counter.
I think we need the cpumask to identify the default logical processor which do counting.Did you restrict these events to system-wide mode only?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Ok, so that means that your cpumask includes one HT per physical core. But then, the result is not the simple aggregation of all the N/2 CPUs.
The counter counts per physical core. The result is the aggregation of all HT cpus in same physical core.
But then don't you need to divide by 2 to get a meaningful result?
Rethink of it. I think I was unclear about the aggregation of all HT cpus in same physical core.
Quoted 8 lines
physical core Cstate should equal to min(logical core C-state). So only all logical core enters C6-state, the physical core enters C6-state, then
CORE_C6_RESIDENCYcounts.So if we only count on one logical core/HT for
CORE_C6_RESIDENCY. We don't need to divide by 2. The count result is the residency when all logical core in C6 (some may deeper).
Ok and here you are assuming you are only measuring one logical CPU per physical core. If this is the case, then I think you are alright. But I wonder what
you'd get when perf stat -a aggregates across all measured CPUs, i.e., one CPU per core.
Quoted 38 lines
Quoted 20 lines
Quoted 20 lines · Kan
Quoted 17 lines
Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my
understanding.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
CORE_C*_RESIDENCY are available for physical processor core. So logical processor in same physical processor core share the same counter.
I think we need the cpumask to identify the default logical processor which do counting.Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines
Did you restrict these events to system-wide mode only?
Ok, so that means that your cpumask includes one HT per physical
core.
Quoted 14 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
But then, the result is not the simple aggregation of all the N/2 CPUs.
The counter counts per physical core. The result is the aggregation of all HT cpus in same physical core.
But then don't you need to divide by 2 to get a meaningful result?
Rethink of it. I think I was unclear about the aggregation of all HT cpus in same physical core.
physical core Cstate should equal to min(logical core C-state). So only all logical core enters C6-state, the physical core enters C6-state, then
CORE_C6_RESIDENCYcounts.
Quoted 8 lines
Quoted 4 lines
So if we only count on one logical core/HT for
CORE_C6_RESIDENCY. We don't need to divide by 2. The count result is the residency when all logical core in C6 (some may deeper).Ok and here you are assuming you are only measuring one logical CPU per physical core. If this is the case, then I think you are alright. But I wonder what you'd get when perf stat -a aggregates across all measured CPUs, i.e., one CPU per core.
Just add them all together.
I think we do the same thing for other PMUs as well.
For uncore or rapl, we get meaningful result by applying --per-socket. Here we can use --per-core.
Thanks,
Kan
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 2 lines
I understand that these metrics are useful and needed however if I look at the broader picture I see many PMUs doing similar things or appearing different when they are actually very close. It would be nice to have a more unified approach. You have RAPL (client, server) which appears as the power PMU. You have the PCU uncore on servers which also provides C-state residency info. Yet, all these appear differently and expose events with different names.
I think we could benefit from a more unifie approach here such that you would be able to do
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
$ perf stat -a -e power/c6-residency/, power/energy-pkg/on client and server without having to change the pmu name of the event names.
Yes, I agree. I'll think about it.
Hi Stephane,
I thought more about your suggestion regarding to create a unified power PMU for all related events include RAPL and residency.
It looks we can benefit from a simple unified name, but it also brings too much confusion.
- cstate residency is the time of the core/socket in specific cstate. While RAPL event is the power core/socket which consumed. They have different concepts.
- cstate residency includes both per-core and per-socket events. RAPL events is only per-socket. So the CPU mask is different. It's very confused that the events in same PMU has different CPU mask.
So I think it should be better to use different PMUs for RAPL and residency.
What do you think?
Thanks,
Kan
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 33 lines · Kan
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 2 lines
I understand that these metrics are useful and needed however if I look at the broader picture I see many PMUs doing similar things or appearing different when they are actually very close. It would be nice to have a more unified approach. You have RAPL (client, server) which appears as the power PMU. You have the PCU uncore on servers which also provides C-state residency info. Yet, all these appear differently and expose events with different names.
I think we could benefit from a more unifie approach here such that you would be able to doQuoted 7 lines
Quoted 7 lines
Quoted 4 lines
$ perf stat -a -e power/c6-residency/, power/energy-pkg/on client and server without having to change the pmu name of the event names.
Yes, I agree. I'll think about it.
Hi Stephane,
I thought more about your suggestion regarding to create a unified power PMU for all related events include RAPL and residency.
It looks we can benefit from a simple unified name, but it also brings too much confusion.
- cstate residency is the time of the core/socket in specific cstate. While RAPL event is the power core/socket which consumed. They have different concepts.
- cstate residency includes both per-core and per-socket events. RAPL events is only per-socket. So the CPU mask is different. It's very confused that the events in same PMU has different CPU mask.
Quoted 4 lines
So I think it should be better to use different PMUs for RAPL and residency.
What do you think?
Well, you are maybe confusing events with PMU. If you look at the core PMU, it cover many events measuring vastly different aspects of the core. Some events are per-thread, others are per-core.
Here, I was thinking it would be good to have some power// PMU with many events covering cstate residency, energy consumption. And yes, some events would be per-socket, others per-core.
Quoted 43 lines · Kan
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
I understand that these metrics are useful and needed however if I look at the broader picture I see many PMUs doing similar things or appearing different when they are actually very close. It would be nice to have a more unified approach. You have RAPL (client, server) which appears as the power PMU. You have the PCU uncore
on
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
servers which also provides C-state residency info. Yet, all these appear differently and expose events with different names. I think we could benefit from a more unifie approach here such that you would be able to do
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
$ perf stat -a -e power/c6-residency/, power/energy-pkg/on client and server without having to change the pmu name of the event names.
Yes, I agree. I'll think about it.
Hi Stephane,
Quoted 10 lines
I thought more about your suggestion regarding to create a unified power PMU for all related events include RAPL and residency. It looks we can benefit from a simple unified name, but it also brings too much confusion.
- cstate residency is the time of the core/socket in specific cstate. While RAPL event is the power core/socket which consumed. They have different concepts.
- cstate residency includes both per-core and per-socket events. RAPL events is only per-socket. So the CPU mask is different. It's very confused that the events in same PMU has different CPUmask.
Quoted 2 lines
So I think it should be better to use different PMUs for RAPL and
residency.
Quoted 3 lines
What do you think?
Quoted 7 lines
Well, you are maybe confusing events with PMU. If you look at the core PMU, it cover many events measuring vastly different aspects of the core. Some events are per-thread, others are per-core.
Here, I was thinking it would be good to have some power// PMU with many events covering cstate residency, energy consumption. And yes, some events would be per-socket, others per-core.
So you agree to create two new cstate PMUs (per-core and per-socket) to cover cstate residency?
If so, I will start to implement the V3 version for two new PMUs.
Thanks,
Kan
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 47 lines · Kan
Quoted 47 lines · Kan
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
I understand that these metrics are useful and needed however if I look at the broader picture I see many PMUs doing similar things or appearing different when they are actually very close. It would be nice to have a more unified approach. You have RAPL (client, server) which appears as the power PMU. You have the PCU uncore
on
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
servers which also provides C-state residency info. Yet, all these appear differently and expose events with different names. I think we could benefit from a more unifie approach here such that you would be able to do
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
$ perf stat -a -e power/c6-residency/, power/energy-pkg/on client and server without having to change the pmu name of the event names.
Yes, I agree. I'll think about it.
Hi Stephane,
Quoted 10 lines
I thought more about your suggestion regarding to create a unified power PMU for all related events include RAPL and residency. It looks we can benefit from a simple unified name, but it also brings too much confusion.
- cstate residency is the time of the core/socket in specific cstate. While RAPL event is the power core/socket which consumed. They have different concepts.
- cstate residency includes both per-core and per-socket events. RAPL events is only per-socket. So the CPU mask is different. It's very confused that the events in same PMU has different CPUmask.
Quoted 2 lines
So I think it should be better to use different PMUs for RAPL and
residency.
Quoted 3 lines
What do you think?
Well, you are maybe confusing events with PMU. If you look at the core PMU, it cover many events measuring vastly different aspects of the core. Some events are per-thread, others are per-core.
Quoted 8 lines
Quoted 3 lines
Here, I was thinking it would be good to have some power// PMU with many events covering cstate residency, energy consumption. And yes, some events would be per-socket, others per-core.
So you agree to create two new cstate PMUs (per-core and per-socket) to cover cstate residency?
If so, I will start to implement the V3 version for two new PMUs.
I did not say that. Instead I said there is some benefits in having everything under a power// PMU, including possibly portability to other non x86 architectures.
Quoted 59 lines · Kan
Quoted 3 lines
Quoted 3 lines
On Tue, Aug 25, 2015 at 1:15 PM, Liang, Kan <[email protected]>
Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
I understand that these metrics are useful and needed however if I look at the broader picture I see many PMUs doing similar things or appearing different when they are actually very close. It would be nice to have a more unified approach. You have RAPL (client,
server) which appears as the power PMU. You have the PCUuncore
Quoted 29 lines
Quoted 29 lines
on
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
servers which also provides C-state residency info. Yet, all these appear differently and expose events with different names. I think we could benefit from a more unifie approach here such that you would be able to do
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
$ perf stat -a -e power/c6-residency/, power/energy-pkg/on client and server without having to change the pmu name of the event names.
Yes, I agree. I'll think about it.
Hi Stephane,
Quoted 11 lines
I thought more about your suggestion regarding to create a unified power PMU for all related events include RAPL and residency. It looks we can benefit from a simple unified name, but it also brings too much confusion.
- cstate residency is the time of the core/socket in specific cstate. While RAPL event is the power core/socket which consumed. They have different concepts.
- cstate residency includes both per-core and per-socket events. RAPL events is only per-socket. So the CPU mask is different. It's very confused that the events in same PMU has different
CPUmask.
Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
So I think it should be better to use different PMUs for RAPL and
residency.
Quoted 3 lines
What do you think?
Well, you are maybe confusing events with PMU. If you look at the core PMU, it cover many events measuring vastly different aspects of
the core.
Quoted 6 lines
Quoted 5 lines
Some events are per-thread, others are per-core.
Here, I was thinking it would be good to have some power// PMU with many events covering cstate residency, energy consumption. And yes, some events would be per-socket, others per-core.
Quoted 7 lines
Quoted 4 lines
So you agree to create two new cstate PMUs (per-core and per-socket) to cover cstate residency?
If so, I will start to implement the V3 version for two new PMUs.I did not say that. Instead I said there is some benefits in having everything under a power// PMU, including possibly portability to other non x86 architectures.
But the events in power// PMU will be per-socket or per-core. How should we handle it?
What should we show in cpumask?
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Peter Zijlstra
maintainerQuoted 7 lines · Stephane Eranian
Quoted 4 lines · Peter Zijlstra
Quoted 1 line
+static
cpumask_tpower_cstate_core_cpu_mask;That one typically does not need a cpumask.
You need to pick one CPU out of the multi-core. But it is for client parts thus there is only one socket. At least this is my understanding.
Oh crud, core as in a group of SMT. There's so much core as in logical cpu usage around these days I got confused :/