TopDown metrics support for Icelake
Subsystems (2)
PATCH v3 1/8 perf/x86/intel: Set correct mask for TOPDOWN.SLOTS
In reply to [email protected]From: Kan Liang <[email protected]>
TOPDOWN.SLOTS(0x0400) is not a generic event. It is only available on fixed counter3.
Don't extend its mask to generic counters.
--- arch/x86/events/intel/core.c | 6 ++++-- arch/x86/include/asm/perf_event.h | 5 +++++ 2 files changed, 9 insertions(+), 2 deletions(-)
arch/x86/events/intel/core.c +4 -2
@@ -5081,12 +5081,14 @@ __init int intel_pmu_init(void)if (x86_pmu.event_constraints) {/* -* event on fixed counter2 (REF_CYCLES) only works on this +* event on fixed counter2 (REF_CYCLES) and +* fixed counter3 (TOPDOWN.SLOTS) only work on this * counter, so do not extend mask to generic counters */ for_each_event_constraint (c, x86_pmu.event_constraints) {if (c->cmask ==FIXED_EVENT_FLAGS - && c->idxmsk64 !=INTEL_PMC_MSK_FIXED_REF_CYCLES ) { + && c->idxmsk64 !=INTEL_PMC_MSK_FIXED_REF_CYCLES + && c->idxmsk64 !=INTEL_PMC_MSK_FIXED_SLOTS ) { c->idxmsk64 |= (1ULL << x86_pmu.num_counters) -1 ; } c->idxmsk64 &=
arch/x86/include/asm/perf_event.h +5 -0
@@ -167,6 +167,11 @@ struct x86_pmu_capability {#define INTEL_PMC_IDX_FIXED_REF_CYCLES (INTEL_PMC_IDX_FIXED + 2)#define INTEL_PMC_MSK_FIXED_REF_CYCLES (1ULL << INTEL_PMC_IDX_FIXED_REF_CYCLES) +/* TOPDOWN.SLOTS: */ +#define MSR_ARCH_PERFMON_FIXED_CTR3 0x30c +#define INTEL_PMC_IDX_FIXED_SLOTS (INTEL_PMC_IDX_FIXED + 3) +#define INTEL_PMC_MSK_FIXED_SLOTS (1ULL << INTEL_PMC_IDX_FIXED_SLOTS) +/* * We model BTS tracing as another fixed-mode PMC. *
Signature · 1 line
2.17.1
PATCH v3 4/8 perf/x86/intel: Support per thread RDPMC TopDown metrics
In reply to [email protected]From: Kan Liang <[email protected]>
With Icelake CPUs, the TopDown metrics are directly available as fixed counters and do not require generic counters, which make it possible to measure TopDown per thread/process instead of only per core.
The metrics and slots values have to be saved/restored during context switching.
The saved values are also used as previous values to calculate the delta.
The PERF_METRICS MSR value will be returned if RDPMC metrics events.
--- arch/x86/events/core.c | 5 +- arch/x86/events/intel/core.c | 102 ++++++++++++++++++++++++++++------- include/linux/perf_event.h | 3 ++ 3 files changed, 91 insertions(+), 19 deletions(-)
arch/x86/events/core.c +4 -1
@@ -2150,7 +2150,10 @@ static int x86_pmu_event_idx(struct perf_event *event)if (!(event->hw.flags &PERF_X86_EVENT_RDPMC_ALLOWED ))return 0 ; -if (x86_pmu.num_counters_fixed && idx >=INTEL_PMC_IDX_FIXED ) { +/* Return PERF_METRICS MSR value for metrics event */ +if (is_metric_idx (idx)) + idx =1 <<29 ; +else if (x86_pmu.num_counters_fixed && idx >=INTEL_PMC_IDX_FIXED ) { idx -=INTEL_PMC_IDX_FIXED ; idx |=1 <<30 ; }
arch/x86/events/intel/core.c +84 -18
145 lines not on this page · raw message
include/linux/perf_event.h +3 -0
@@ -133,6 +133,9 @@ struct hw_perf_event {struct hw_perf_event_extra extra_reg;struct hw_perf_event_extra branch_reg; + +u64 saved_slots; +u64 saved_metric; };struct {/* software */ struct hrtimer hrtimer;
Signature · 1 line
2.17.1
PATCH v3 8/8 perf, tools: Add documentation for topdown metrics
In reply to [email protected]From: Andi Kleen <[email protected]>
Add some documentation how to use the topdown metrics in ring 3.
--- tools/perf/Documentation/topdown.txt | 223 +++++++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 tools/perf/Documentation/topdown.txt
tools/perf/Documentation/topdown.txt new file +223 -0
@@ -0,0 +1,223 @@ +Using TopDown metrics in user space +----------------------------------- + +Intel CPUs (since Sandy Bridge and Silvermont) support a TopDown +methology to break down CPU pipeline execution into 4 bottlenecks: +frontend bound, backend bound, bad speculation, retiring. + +For more details on Topdown see [1][5] + +Traditionally this was implemented by events in generic counters +and specific formulas to compute the bottlenecks. + +perf stat --topdown implements this. + +% perf stat -a --topdown -I1000 +# time counts unit events + 1.000373951 8,460,978,609 topdown-retiring # 22.9% retiring + 1.000373951 3,445,383,303 topdown-bad-spec # 9.3% bad speculation + 1.000373951 15,886,483,355 topdown-fe-bound # 43.0% frontend bound + 1.000373951 9,163,488,720 topdown-be-bound # 24.8% backend bound + 2.000782154 8,477,925,431 topdown-retiring # 22.9% retiring + 2.000782154 3,459,151,256 topdown-bad-spec # 9.3% bad speculation + 2.000782154 15,947,224,725 topdown-fe-bound # 43.0% frontend bound + 2.000782154 9,145,551,695 topdown-be-bound # 24.7% backend bound + 3.001155967 8,487,323,125 topdown-retiring # 22.9% retiring + 3.001155967 3,451,808,066 topdown-bad-spec # 9.3% bad speculation + 3.001155967 15,959,068,902 topdown-fe-bound # 43.0% frontend bound + 3.001155967 9,172,479,784 topdown-be-bound # 24.7% backend bound +... + +Full Top Down includes more levels that can break down the +bottlenecks further. This is not directly implemented in perf, +but available in other tools that can run on top of perf, +such as toplev[2] or vtune[3] + +New Topdown features in Icelake +=============================== + +With Icelake CPUs the TopDown metrics are directly available as +fixed counters and do not require generic counters. This allows +to collect TopDown always in addition to other events. + +This also enables measuring TopDown per thread/process instead +of only per core. + +Using TopDown through RDPMC in applications on Icelake +====================================================== + +For more fine grained measurements it can be useful to +access the new directly from user space. This is more complicated, +but drastically lowers overhead. + +On Icelake, there is a new fixed counter 3: SLOTS, which reports +"pipeline SLOTS" (cycles multiplied by core issue width) and a +metric register that reports slots ratios for the different bottleneck +categories. + +The metrics counter is CPU model specific and is not be available +on older CPUs. + +Example code +============ + +Library functions to do the functionality described below +is also available in libjevents [4] + +The application opens a perf_event file descriptor +and sets up fixed counter 3 (SLOTS) to start and +allow user programs to read the performance counters. + +Fixed counter 3 is mapped to a pseudo event event=0x00, umask=04, +so the perf_event_attr structure should be initialized with +{ .config = 0x0400, .type = PERF_TYPE_RAW } + +#include <linux/perf_event.h> +#include <sys/syscall.h> +#include <unistd.h> + +/* Provide own perf_event_open stub because glibc doesn't */ +__attribute__((weak)) +int perf_event_open(struct perf_event_attr *attr, pid_t pid, + int cpu, int group_fd, unsigned long flags) +{ + return syscall(__NR_perf_event_open, attr, pid, cpu, group_fd, flags); +} + +/* open slots counter file descriptor for current task */ +struct perf_event_attr slots = { + .type = PERF_TYPE_RAW, + .size = sizeof(struct perf_event_attr), + .config = 0x400, + .exclude_kernel = 1, +}; + +int fd = perf_event_open(&slots, 0, -1, -1, 0); +if (fd < 0) + ... error ... + +The RDPMC instruction (or _rdpmc compiler intrinsic) can now be used +to read slots and the topdown metrics at different points of the program: + +#include <stdint.h> +#include <x86intrin.h> + +#define RDPMC_FIXED (1 << 30) /* return fixed counters */ +#define RDPMC_METRIC (1 << 29) /* return metric counters */ + +#define FIXED_COUNTER_SLOTS 3 +#define METRIC_COUNTER_TOPDOWN_L1 0 + +static inline uint64_t read_slots(void) +{ + return _rdpmc(RDPMC_FIXED | FIXED_COUNTER_SLOTS); +} + +static inline uint64_t read_metrics(void) +{ + return _rdpmc(RDPMC_METRIC | METRIC_COUNTER_TOPDOWN_L1); +} + +Then the program can be instrumented to read these metrics at different +points. + +It's not a good idea to do this with too short code regions, +as the parallelism and overlap in the CPU program execution will +cause too much measurement inaccuracy. For example instrumenting +individual basic blocks is definitely too fine grained. + +Decoding metrics values +======================= + +The value reported by read_metrics() contains four 8 bit fields +that represent a scaled ratio that represent the Level 1 bottleneck. +All four fields add up to 0xff (= 100%) + +The binary ratios in the metric value can be converted to float ratios: + +#define GET_METRIC(m, i) (((m) >> (i*8)) & 0xff) + +#define TOPDOWN_RETIRING(val) ((float)GET_METRIC(val, 0) / 0xff) +#define TOPDOWN_BAD_SPEC(val) ((float)GET_METRIC(val, 1) / 0xff) +#define TOPDOWN_FE_BOUND(val) ((float)GET_METRIC(val, 2) / 0xff) +#define TOPDOWN_BE_BOUND(val) ((float)GET_METRIC(val, 3) / 0xff) + +and then converted to percent for printing. + +The ratios in the metric accumulate for the time when the counter +is enabled. For measuring programs it is often useful to measure +specific sections. For this it is needed to deltas on metrics. + +This can be done by scaling the metrics with the slots counter +read at the same time. + +Then it's possible to take deltas of these slots counts +measured at different points, and determine the metrics +for that time period. + + slots_a = read_slots(); + metric_a = read_metrics(); + + ... larger code region ... + + slots_b = read_slots() + metric_b = read_metrics() + + # compute scaled metrics for measurement a + retiring_slots_a = GET_METRIC(metric_a, 0) * slots_a + bad_spec_slots_a = GET_METRIC(metric_a, 1) * slots_a + fe_bound_slots_a = GET_METRIC(metric_a, 2) * slots_a + be_bound_slots_a = GET_METRIC(metric_a, 3) * slots_a + + # compute delta scaled metrics between b and a + retiring_slots = GET_METRIC(metric_b, 0) * slots_b - retiring_slots_a + bad_spec_slots = GET_METRIC(metric_b, 1) * slots_b - bad_spec_slots_a + fe_bound_slots = GET_METRIC(metric_b, 2) * slots_b - fe_bound_slots_a + be_bound_slots = GET_METRIC(metric_b, 3) * slots_b - be_bound_slots_a + +Later the individual ratios for the measurement period can be recreated +from these counts. + + slots_delta = slots_b - slots_a + retiring_ratio = (float)retiring_slots / slots_delta + bad_spec_ratio = (float)bad_spec_slots / slots_delta + fe_bound_ratio = (float)fe_bound_slots / slots_delta + be_bound_ratio = (float)be_bound_slots / slota_delta + + printf("Retiring %.2f%% Bad Speculation %.2f%% FE Bound %.2f%% BE Bound %.2f%%\n", + retiring_ratio * 100., + bad_spec_ratio * 100., + fe_bound_ratio * 100., + be_bound_ratio * 100.); + +Resetting metrics counters +========================== + +Since the individual metrics are only 8bit they lose precision for +short regions over time because the number of cycles covered by each +fraction bit shrinks. So the counters need to be reset regularly. + +When using the kernel perf API the kernel resets on every read. +So as long as the reading is at reasonable intervals (every few +seconds) the precision is good. + +When using perf stat it is recommended to always use the -I option, +with no longer interval than a few seconds + + perf stat -I 1000 --topdown ... + +For user programs using RDPMC directly the counter can +be reset explicitly using ioctl: + + ioctl(perf_fd, PERF_EVENT_IOC_RESET, 0); + +This "opens" a new measurement period. + +A program using RDPMC for TopDown should schedule such a reset +regularly, as in every few seconds. + +[1] https://software.intel.com/en-us/top-down-microarchitecture-analysis-method-win +[2] https://github.com/andikleen/pmu-tools/wiki/toplev-manual +[3] https://software.intel.com/en-us/intel-vtune-amplifier-xe +[4] https://github.com/andikleen/pmu-tools/tree/master/jevents +[5] https://sites.google.com/site/analysismethods/yasin-pubs
Signature · 1 line
2.17.1
PATCH v3 6/8 perf/x86/intel: Disable sampling read slots and topdown
In reply to [email protected]From: Kan Liang <[email protected]>
The slots event supports sampling. Users may sampling read slots and metrics events, e.g perf record -e '{slots, topdown-retiring}:S'. But the metrics event will reset the fixed counter 3 which will impact the sampling of the slots event.
Add specific validate_group() support to reject the case and error out for Icelake.
An alternative fix may unconditionally disable SLOTS sampling. But it's not a decent fix. Because users may want to only sampling slot events without topdown metrics event.
--- arch/x86/events/core.c | 4 ++++ arch/x86/events/intel/core.c | 20 ++++++++++++++++++++ arch/x86/events/perf_event.h | 2 ++ 3 files changed, 26 insertions(+)
arch/x86/events/core.c +4 -0
@@ -2057,7 +2057,11 @@ static int validate_group(struct perf_event *event) fake_cpuc->n_events =0 ; ret = x86_pmu.schedule_events (fake_cpuc, n,NULL ); +if (ret) + goto out; +if (x86_pmu.validate_group) + ret = x86_pmu.validate_group (fake_cpuc, n); out:free_fake_cpuc (fake_cpuc);return ret;
arch/x86/events/intel/core.c +20 -0
@@ -4489,6 +4489,25 @@ static __init void intel_ht_bug(void) x86_pmu.stop_scheduling = intel_stop_scheduling; } +static int icl_validate_group (struct cpu_hw_events *cpuc,int n) +{ +bool has_sampling_slots = false, has_metrics = false; +struct perf_event *e; +int i; + +for (i =0 ; i < n; i++) { + e = cpuc->event_list[i]; +if (is_slots_event (e) &&is_sampling_event (e)) + has_sampling_slots = true; + +if (is_metric_event (e)) + has_metrics = true; + } +if (unlikely (has_sampling_slots && has_metrics)) +return -EINVAL ; +return 0 ; +} +EVENT_ATTR_STR (mem-loads, mem_ld_hsw,"event=0xcd,umask=0x1,ldlat=3" );EVENT_ATTR_STR (mem-stores, mem_st_hsw,"event=0xd0,umask=0x82" )
@@ -5334,6 +5353,7 @@ __init int intel_pmu_init(void)intel_pmu_pebs_data_source_skl (pmem); x86_pmu.update_topdown_event = icl_update_topdown_event; x86_pmu.set_topdown_event_period = icl_set_topdown_event_period; + x86_pmu.validate_group = icl_validate_group;pr_cont ("Icelake events, " ); name ="icelake" ;break ;
arch/x86/events/perf_event.h +2 -0
@@ -629,6 +629,8 @@ struct x86_pmu {int perfctr_second_write;u64 (*limit_period)(struct perf_event *event,u64 l); +int (*validate_group)(struct cpu_hw_events *cpuc,int n); +/* PMI handler bits */ unsigned int late_ack :1, counter_freezing :1;
Signature · 1 line
2.17.1
PATCH v3 7/8 perf, tools, stat: Support new per thread TopDown metrics
In reply to [email protected]From: Andi Kleen <[email protected]>
Icelake has support for reporting per thread TopDown metrics. These are reported differently than the previous TopDown support, each metric is standalone, but scaled to pipeline "slots". We don't need to do anything special for HyperThreading anymore. Teach perf stat --topdown to handle these new metrics and print them in the same way as the previous TopDown metrics. The restrictions of only being able to report information per core is gone.
--- tools/perf/Documentation/perf-stat.txt | 9 ++- tools/perf/builtin-stat.c | 24 +++++++ tools/perf/util/stat-shadow.c | 89 ++++++++++++++++++++++++++ tools/perf/util/stat.c | 4 ++ tools/perf/util/stat.h | 8 +++ 5 files changed, 132 insertions(+), 2 deletions(-)
tools/perf/Documentation/perf-stat.txt +7 -2
@@ -279,8 +279,13 @@ if the workload is actually bound by the CPU and not by something else. For best results it is usually a good idea to use it with interval mode like -I 1000, as the bottleneck of workloads can change often. -The top down metrics are collected per core instead of per -CPU thread. Per core mode is automatically enabled +This enables --metric-only, unless overridden with --no-metric-only. + +The following restrictions only apply to older Intel CPUs and Atom, +on newer CPUs (IceLake and later) TopDown can be collected for any thread: + +The top down metrics are collected per core instead of per CPU thread. +Per core mode is automatically enabled and -a (global monitoring) is needed, requiring root rights or perf.perf_event_paranoid=-1.
tools/perf/builtin-stat.c +24 -0
@@ -122,6 +122,14 @@ static const char * topdown_attrs[] = { NULL, }; +static const char *topdown_metric_attrs[] = { +"topdown-retiring" , +"topdown-bad-spec" , +"topdown-fe-bound" , +"topdown-be-bound" , +NULL , +}; +static const char *smi_cost_attrs = {"{" "msr/aperf/,"
@@ -1323,6 +1331,21 @@ static int add_default_attributes(void)char *str =NULL ;bool warn = false; +if (topdown_filter_events (topdown_metric_attrs, &str,1 ) <0 ) { +pr_err ("Out of memory\n" ); +return -1 ; + } +if (topdown_metric_attrs[0 ] && str) { +if (!stat_config.interval) { +fprintf (stat_config.output, +"Topdown accuracy may decreases when measuring long period.\n" +"Please print the result regularly, e.g. -I1000\n" ); + } + goto setup_metrics; + } + + str =NULL ; +if (stat_config.aggr_mode !=AGGR_GLOBAL && stat_config.aggr_mode !=AGGR_CORE ) {pr_err ("top down event configuration requires --per-core mode\n" );
@@ -1344,6 +1367,7 @@ static int add_default_attributes(void)if (topdown_attrs[0 ] && str) {if (warn)arch_topdown_group_warn (); +setup_metrics: err =parse_events (evsel_list, str, &errinfo);if (err) { fprintf(stderr,
tools/perf/util/stat-shadow.c +89 -0
107 lines not on this page · raw message
tools/perf/util/stat.c +4 -0
@@ -88,6 +88,10 @@ static const char *id_str[PERF_STAT_EVSEL_ID__MAX] = {ID (TOPDOWN_SLOTS_RETIRED , topdown-slots-retired),ID (TOPDOWN_FETCH_BUBBLES , topdown-fetch-bubbles),ID (TOPDOWN_RECOVERY_BUBBLES , topdown-recovery-bubbles), +ID (TOPDOWN_RETIRING , topdown-retiring), +ID (TOPDOWN_BAD_SPEC , topdown-bad-spec), +ID (TOPDOWN_FE_BOUND , topdown-fe-bound), +ID (TOPDOWN_BE_BOUND , topdown-be-bound), ID(SMI_NUM, msr/smi/), ID(APERF, msr/aperf/), };
tools/perf/util/stat.h +8 -0
@@ -29,6 +29,10 @@ enum perf_stat_evsel_id {PERF_STAT_EVSEL_ID__TOPDOWN_SLOTS_RETIRED ,PERF_STAT_EVSEL_ID__TOPDOWN_FETCH_BUBBLES ,PERF_STAT_EVSEL_ID__TOPDOWN_RECOVERY_BUBBLES , +PERF_STAT_EVSEL_ID__TOPDOWN_RETIRING , +PERF_STAT_EVSEL_ID__TOPDOWN_BAD_SPEC , +PERF_STAT_EVSEL_ID__TOPDOWN_FE_BOUND , +PERF_STAT_EVSEL_ID__TOPDOWN_BE_BOUND ,PERF_STAT_EVSEL_ID__SMI_NUM ,PERF_STAT_EVSEL_ID__APERF , PERF_STAT_EVSEL_ID__MAX,
@@ -82,6 +86,10 @@ enum stat_type {STAT_TOPDOWN_SLOTS_RETIRED ,STAT_TOPDOWN_FETCH_BUBBLES ,STAT_TOPDOWN_RECOVERY_BUBBLES , +STAT_TOPDOWN_RETIRING , +STAT_TOPDOWN_BAD_SPEC , +STAT_TOPDOWN_FE_BOUND , +STAT_TOPDOWN_BE_BOUND ,STAT_SMI_NUM ,STAT_APERF ,STAT_MAX
Signature · 1 line
2.17.1
PATCH v3 5/8 perf/x86/intel: Export TopDown events for Icelake
In reply to [email protected]From: Kan Liang <[email protected]>
Export new TopDown metrics events for perf that map to the sub metrics in the metrics register, and another for the new slots fixed counter. This makes the new fixed counters in Icelake visible to the perf user tools.
--- arch/x86/events/intel/core.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arch/x86/events/intel/core.c +16 -0
@@ -321,6 +321,12 @@ EVENT_ATTR_STR_HT(topdown-recovery-bubbles, td_recovery_bubbles,EVENT_ATTR_STR_HT (topdown-recovery-bubbles.scale, td_recovery_bubbles_scale,"4" ,"2" ); +EVENT_ATTR_STR (slots, slots,"event=0x00,umask=0x4" ); +EVENT_ATTR_STR (topdown-retiring, td_retiring,"event=0x00,umask=0x10" ); +EVENT_ATTR_STR (topdown-bad-spec, td_bad_spec,"event=0x00,umask=0x11" ); +EVENT_ATTR_STR (topdown-fe-bound, td_fe_bound,"event=0x00,umask=0x12" ); +EVENT_ATTR_STR (topdown-be-bound, td_be_bound,"event=0x00,umask=0x13" ); +static struct attribute *snb_events_attrs[] = {EVENT_PTR (td_slots_issued),EVENT_PTR (td_slots_retired),
@@ -4544,6 +4550,15 @@ static struct attribute *icl_events_attrs[] = { NULL, }; +static struct attribute *icl_td_events_attrs[] = { +EVENT_PTR (slots), +EVENT_PTR (td_retiring), +EVENT_PTR (td_bad_spec), +EVENT_PTR (td_fe_bound), +EVENT_PTR (td_be_bound), +NULL , +}; +static struct attribute *icl_tsx_events_attrs[] = {EVENT_PTR (tx_start),EVENT_PTR (tx_abort),
@@ -5312,6 +5327,7 @@ __init int intel_pmu_init(void) hsw_format_attr : nhm_format_attr; extra_skl_attr = skl_format_attr; mem_attr = icl_events_attrs; + td_attr = icl_td_events_attrs; tsx_attr = icl_tsx_events_attrs; x86_pmu.rtm_abort_event = X86_CONFIG(.event=0xca, .umask=0x02); x86_pmu.lbr_pt_coexist = true;
Signature · 1 line
2.17.1
PATCH v3 3/8 perf/x86/intel: Support hardware TopDown metrics
In reply to [email protected]From: Kan Liang <[email protected]>
Intro
=====
Icelake has support for measuring the four top level TopDown metrics directly in hardware. This is implemented by an additional "metrics" register, and a new Fixed Counter 3 that measures pipeline "slots".
Events
======
We export four metric events as separate perf events, which map to internal "metrics" counter register. Those events do not exist in hardware, but can be allocated by the scheduler.
For the event mapping we use a special 0x00 event code, which is reserved for fake events. The metric events start from umask 0x10.
When setting up such events they point to the slots counter, and a special callback, update_topdown_event(), reads the additional metrics msr to generate the metrics. Then the metric is reported by multiplying the metric (percentage) with slots.
This multiplication allows to easily keep a running count, for example when the slots counter overflows, and makes all the standard tools, such as a perf stat, work. They can do deltas of the values without needing to know about percentages. This also simplifies accumulating the counts of child events, which otherwise would need to know how to average percent values.
All four metric events don't support sampling. Since they will be handled specially for event update, a flag PERF_X86_EVENT_TOPDOWN is introduced to indicate this case.
The slots event can support both sampling and counting. For counting, the flag is also applied.
For sampling, it will be handled normally as other normal events.
Groups
======
To avoid reading the METRICS register multiple times, the metrics and slots value can only be updated by the first slots/metrics event in a group. All active slots and metrics events will be updated one time.
Reset
======
The PERF_METRICS and Fixed counter 3 have to be reset for each read, because:
- The 8bit metrics ratio values lose precision when the measurement period gets longer.
- The PERF_METRICS may report wrong value if its delta was less than 1/255 of SLOTS (Fixed counter 3).
Also, for counting, the -max_period is the initial value of the SLOTS. The huge initial value will definitely trigger the issue mentioned above. Force initial value as 0 for topdown and slots event counting.
NMI
======
The METRICS register may be overflow. The bit 48 of STATUS register will be set. If so, update all active slots and metrics events.
The update_topdown_event() has to read two registers separately. The values may be modify by a NMI. PMU has to be disabled before calling the function.
RDPMC
======
RDPMC is temporarily disabled. The following patch will enable it.
--- arch/x86/events/core.c | 10 ++ arch/x86/events/intel/core.c | 230 ++++++++++++++++++++++++++++++- arch/x86/events/perf_event.h | 17 +++ arch/x86/include/asm/msr-index.h | 2 + 4 files changed, 255 insertions(+), 4 deletions(-)
arch/x86/events/core.c +10 -0
@@ -76,6 +76,8 @@ u64 x86_perf_event_update(struct perf_event *event)if (idx ==INTEL_PMC_IDX_FIXED_BTS )return 0 ; +if (is_topdown_count (event) && x86_pmu.update_topdown_event) +return x86_pmu.update_topdown_event (event);/* * Careful: an NMI might modify the previous event value. *
@@ -1003,6 +1005,10 @@ static int collect_events(struct cpu_hw_events *cpuc, struct perf_event *leader, max_count = x86_pmu.num_counters + x86_pmu.num_counters_fixed; +/* There are 4 TopDown metrics events. */ +if (x86_pmu.intel_cap.perf_metrics) + max_count +=4 ; +/* current number of events already accepted */ n = cpuc->n_events;
@@ -1184,6 +1190,10 @@ int x86_perf_event_set_period(struct perf_event *event)if (idx ==INTEL_PMC_IDX_FIXED_BTS )return 0 ; +if (unlikely (is_topdown_count (event)) && + x86_pmu.set_topdown_event_period) +return x86_pmu.set_topdown_event_period (event); +/* * If we are way outside a reasonable range then just skip forward: */
arch/x86/events/intel/core.c +226 -4
282 lines not on this page · raw message
arch/x86/events/perf_event.h +17 -0
@@ -40,6 +40,7 @@ enum extra_reg_type {EXTRA_REG_LBR =2 ,/* lbr_select */ EXTRA_REG_LDLAT =3 ,/* ld_lat_threshold */ EXTRA_REG_FE =4 ,/* fe_* */ +EXTRA_REG_TOPDOWN =5 ,/* Topdown slots/metrics */ EXTRA_REG_MAX /* number of entries needed */ };
@@ -76,6 +77,12 @@ static inline bool constraint_match(struct event_constraint *c, u64 ecode)#define PERF_X86_EVENT_EXCL_ACCT 0x0100/* accounted EXCL event */ #define PERF_X86_EVENT_AUTO_RELOAD 0x0200/* use PEBS auto-reload */ #define PERF_X86_EVENT_LARGE_PEBS 0x0400/* use large PEBS */ +#define PERF_X86_EVENT_TOPDOWN 0x0800/* Count Topdown slots/metrics events */ + +static inline bool is_topdown_count (struct perf_event *event) +{ +return event->hw.flags &PERF_X86_EVENT_TOPDOWN ; +}struct amd_nb {int nb_id;/* NorthBridge id */
@@ -509,6 +516,9 @@ struct extra_reg { 0xffff, \ LDLAT) +#define INTEL_UEVENT_TOPDOWN_EXTRA_REG (event) \ + EVENT_EXTRA_REG(event, 0, 0xfcff, -1L, TOPDOWN) +#define EVENT_EXTRA_END EVENT_EXTRA_REG(0, 0, 0, 0, RSP_0)union perf_capabilities {
@@ -524,6 +534,7 @@ union perf_capabilities {*/ u64 full_width_write:1; u64 pebs_baseline:1; + u64 perf_metrics:1; };u64 capabilities; };
@@ -686,6 +697,12 @@ struct x86_pmu {*/ atomic_t lbr_exclusive[x86_lbr_exclusive_max]; +/* +* Intel perf metrics +*/ +u64 (*update_topdown_event)(struct perf_event *event); +int (*set_topdown_event_period)(struct perf_event *event); +/* * AMD bits */
arch/x86/include/asm/msr-index.h +2 -0
@@ -118,6 +118,8 @@#define MSR_TURBO_RATIO_LIMIT1 0x000001ae#define MSR_TURBO_RATIO_LIMIT2 0x000001af +#define MSR_PERF_METRICS 0x00000329 +#define MSR_LBR_SELECT 0x000001c8#define MSR_LBR_TOS 0x000001c9#define MSR_LBR_NHM_FROM 0x00000680
Signature · 1 line
2.17.1
PATCH v3 2/8 perf/x86/intel: Basic support for metrics counters
In reply to [email protected]From: Kan Liang <[email protected]>
Metrics counters (hardware counters containing multiple metrics) are modeled as separate registers for each TopDown metric events, with an extra reg being used for coordinating access to the underlying register in the scheduler.
Adds the basic infrastructure to separate the scheduler register indexes from the actual hardware register indexes. In most cases the MSR address is already used correctly, but for code using indexes we need calculate the correct underlying register.
The TopDown metric events share the fixed counter 3. It only needs enable/disable once for them.
Move BTS index to 47. Because bit 48 in the PERF_GLOBAL_STATUS is used to indicate the overflow status of PERF_METRICS counters now.
Naming:
The events which uses Metrics counters are called TopDown metric events or metric events in the code.
The fixed counter 3 is called TopDown slots event or slots event. Topdown events stand for metric events + slots event in the code.
--- arch/x86/events/core.c | 16 ++++++++-- arch/x86/events/intel/core.c | 28 ++++++++++++------ arch/x86/events/perf_event.h | 14 +++++++++ arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/perf_event.h | 49 +++++++++++++++++++++++++++++-- 5 files changed, 95 insertions(+), 13 deletions(-)
arch/x86/events/core.c +14 -2
@@ -1033,18 +1033,30 @@ static inline void x86_assign_hw_event(struct perf_event *event, struct cpu_hw_events *cpuc, int i) {struct hw_perf_event *hwc = &event->hw; +int reg_idx; hwc->idx = cpuc->assign[i]; hwc->last_cpu =smp_processor_id (); hwc->last_tag = ++cpuc->tags[i]; +/* +* Metrics counters use different indexes in the scheduler +* versus the hardware. +* +* Map metrics to fixed counter 3 (which is the base count), +* but the update event callback reads the extra metric register +* and converts to the right metric. +*/ + reg_idx =get_reg_idx (hwc->idx); +if (hwc->idx ==INTEL_PMC_IDX_FIXED_BTS ) { hwc->config_base =0 ; hwc->event_base =0 ; }else if (hwc->idx >=INTEL_PMC_IDX_FIXED ) { hwc->config_base =MSR_ARCH_PERFMON_FIXED_CTR_CTRL ; - hwc->event_base =MSR_ARCH_PERFMON_FIXED_CTR0 + (hwc->idx -INTEL_PMC_IDX_FIXED ); - hwc->event_base_rdpmc = (hwc->idx -INTEL_PMC_IDX_FIXED ) |1 <<30 ; + hwc->event_base =MSR_ARCH_PERFMON_FIXED_CTR0 + + (reg_idx -INTEL_PMC_IDX_FIXED ); + hwc->event_base_rdpmc = (reg_idx -INTEL_PMC_IDX_FIXED ) |1 <<30 ; }else { hwc->config_base =x86_pmu_config_addr (hwc->idx); hwc->event_base =x86_pmu_event_addr (hwc->idx);
arch/x86/events/intel/core.c +19 -9
@@ -2131,7 +2131,7 @@ static inline void intel_pmu_ack_status(u64 ack)static void intel_pmu_disable_fixed (struct hw_perf_event *hwc) { -int idx = hwc->idx -INTEL_PMC_IDX_FIXED ; +int idx =get_reg_idx (hwc->idx) -INTEL_PMC_IDX_FIXED ;u64 ctrl_val, mask; mask =0xfULL << (idx *4 );
@@ -2150,6 +2150,7 @@ static void intel_pmu_disable_event(struct perf_event *event) {struct hw_perf_event *hwc = &event->hw;struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +int reg_idx =get_reg_idx (hwc->idx);if (unlikely (hwc->idx ==INTEL_PMC_IDX_FIXED_BTS )) {intel_pmu_disable_bts ();
@@ -2157,9 +2158,16 @@ static void intel_pmu_disable_event(struct perf_event *event)return ; } - cpuc->intel_ctrl_guest_mask &= ~(1ull << hwc->idx); - cpuc->intel_ctrl_host_mask &= ~(1ull << hwc->idx); - cpuc->intel_cp_status &= ~(1ull << hwc->idx); +/* +* When any other topdown events are still enabled, +* cancel the disabling. +*/ +if (has_other_topdown_event (cpuc->active_mask, hwc->idx)) +return ; + + cpuc->intel_ctrl_guest_mask &= ~(1ull << reg_idx); + cpuc->intel_ctrl_host_mask &= ~(1ull << reg_idx); + cpuc->intel_cp_status &= ~(1ull << reg_idx);if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL ))intel_pmu_disable_fixed (hwc);
@@ -2193,7 +2201,7 @@ static void intel_pmu_read_event(struct perf_event *event)static void intel_pmu_enable_fixed (struct perf_event *event) {struct hw_perf_event *hwc = &event->hw; -int idx = hwc->idx -INTEL_PMC_IDX_FIXED ; +int idx =get_reg_idx (hwc->idx) -INTEL_PMC_IDX_FIXED ;u64 ctrl_val, mask, bits =0 ;/*
@@ -2232,6 +2240,7 @@ static void intel_pmu_enable_event(struct perf_event *event) {struct hw_perf_event *hwc = &event->hw;struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +int reg_idx =get_reg_idx (hwc->idx);if (unlikely (hwc->idx ==INTEL_PMC_IDX_FIXED_BTS )) {if (!__this_cpu_read (cpu_hw_events.enabled))
@@ -2242,18 +2251,19 @@ static void intel_pmu_enable_event(struct perf_event *event) }if (event->attr.exclude_host) - cpuc->intel_ctrl_guest_mask |= (1ull << hwc->idx); + cpuc->intel_ctrl_guest_mask |= (1ull << reg_idx);if (event->attr.exclude_guest) - cpuc->intel_ctrl_host_mask |= (1ull << hwc->idx); + cpuc->intel_ctrl_host_mask |= (1ull << reg_idx);if (unlikely (event_is_checkpointed (event))) - cpuc->intel_cp_status |= (1ull << hwc->idx); + cpuc->intel_cp_status |= (1ull << reg_idx);if (unlikely (event->attr.precise_ip))intel_pmu_pebs_enable (event);if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL )) { -intel_pmu_enable_fixed (event); +if (!has_other_topdown_event (cpuc->active_mask, hwc->idx)) +intel_pmu_enable_fixed (event);return ; }
arch/x86/events/perf_event.h +14 -0
@@ -348,6 +348,20 @@ struct cpu_hw_events {#define FIXED_EVENT_CONSTRAINT (c, n) \ EVENT_CONSTRAINT(c, (1ULL << (32+n)), FIXED_EVENT_FLAGS) +/* +* Special metric counters do not actually exist, but get remapped +* to a combination of FxCtr3 + MSR_PERF_METRICS +* +* This allocates them to a dummy offset for the scheduler. +* This does not allow sharing of multiple users of the same +* metric without multiplexing, even though the hardware supports that +* in principle. +*/ + +#define METRIC_EVENT_CONSTRAINT (c, n) \ + EVENT_CONSTRAINT(c, (1ULL << (INTEL_PMC_IDX_FIXED_METRIC_BASE+n)), \ + FIXED_EVENT_FLAGS) +/* * Constraint on the Event code + UMask */
arch/x86/include/asm/msr-index.h +1 -0
@@ -793,6 +793,7 @@#define MSR_CORE_PERF_FIXED_CTR0 0x00000309#define MSR_CORE_PERF_FIXED_CTR1 0x0000030a#define MSR_CORE_PERF_FIXED_CTR2 0x0000030b +#define MSR_CORE_PERF_FIXED_CTR3 0x0000030c#define MSR_CORE_PERF_FIXED_CTR_CTRL 0x0000038d#define MSR_CORE_PERF_GLOBAL_STATUS 0x0000038e#define MSR_CORE_PERF_GLOBAL_CTRL 0x0000038f
arch/x86/include/asm/perf_event.h +47 -2
58 lines not on this page · raw message
Signature · 1 line
2.17.1
v3 cover TopDown metrics support for Icelake
From: Kan Liang <[email protected]>
Icelake has support for measuring the level 1 TopDown metrics directly in hardware. This is implemented by an additional METRICS register, and a new Fixed Counter 3 that measures pipeline SLOTS.
Four TopDown metric events as separate perf events, which map to internal METRICS register, are exposed. They are topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be-bound. Those events do not exist in hardware, but can be allocated by the scheduler. The value of TopDown metric events can be calculated by multiplying the METRICS (percentage) register with SLOTS fixed counter.
New in Icelake
- Do not require generic counters. This allows to collect TopDown always in addition to other events.
- Measuring TopDown per thread/process instead of only per core
Limitation
- To get accurate result and avoid reading the METRICS register multiple times, the TopDown metrics events and SLOTS event have to be in the same group.
- METRICS and SLOTS registers have to be cleared after each read by SW. That is to prevent the lose of precision and a known side effect of METRICS register.
- Cannot do sampling read SLOTS and TopDown metric events
Please refer SDM Vol3, 18.3.9.3 Performance Metrics for the details of TopDown metrics.
Changes since V2:
- Rebase on top of v5.3-rc1
Key changes since V1:
- Remove variables for reg_idx and enabled_events[] array. The reg_idx can be calculated by idx in runtime.
Using existing active_mask to replace enabled_events.
- Choose value 47 for the fixed index of BTS.
- Support OVF_PERF_METRICS overflow bit in PMI handler
- Drops the caching mechanism and related variables
New mechanism is to update all active slots/metrics events for the first slots/metrics events in a group. For each group reading, it still only read the slots/perf_metrics MSR once
- Disable PMU for read of topdown events to avoid the NMI issue
- Move RDPMC support to a separate patch
- Using event=0x00,umask=0x1X for topdown metrics events
- Drop the patch which add REMOVE transaction
We can indicate x86_pmu_stop() by checking
(event && !test_bit(event->hw.idx, cpuc->active_mask)), which is a good place to save the slots/metrics MSR value
Andi Kleen (2):
perf, tools, stat: Support new per thread TopDown metrics perf, tools: Add documentation for topdown metrics
Kan Liang (6):
perf/x86/intel: Set correct mask for TOPDOWN.SLOTS perf/x86/intel: Basic support for metrics counters perf/x86/intel: Support hardware TopDown metrics perf/x86/intel: Support per thread RDPMC TopDown metrics perf/x86/intel: Export TopDown events for Icelake perf/x86/intel: Disable sampling read slots and topdown
arch/x86/events/core.c | 35 ++- arch/x86/events/intel/core.c | 362 ++++++++++++++++++++++++- arch/x86/events/perf_event.h | 33 +++ arch/x86/include/asm/msr-index.h | 3 + arch/x86/include/asm/perf_event.h | 54 +++- include/linux/perf_event.h | 3 + tools/perf/Documentation/perf-stat.txt | 9 +- tools/perf/Documentation/topdown.txt | 223 +++++++++++++++ tools/perf/builtin-stat.c | 24 ++ tools/perf/util/stat-shadow.c | 89 ++++++ tools/perf/util/stat.c | 4 + tools/perf/util/stat.h | 8 + 12 files changed, 827 insertions(+), 20 deletions(-) create mode 100644 tools/perf/Documentation/topdown.txt
Signature · 1 line
2.17.1
Hi Peter,
Any comments for this series?
Thanks,
Kan
Quoted 78 lines · On 7/24/2019 1:17 PM, [email protected] wrote:
From: Kan Liang <[email protected]>
Icelake has support for measuring the level 1 TopDown metrics directly in hardware. This is implemented by an additional METRICS register, and a new Fixed Counter 3 that measures pipeline SLOTS.
Four TopDown metric events as separate perf events, which map to internal METRICS register, are exposed. They are topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be-bound. Those events do not exist in hardware, but can be allocated by the scheduler. The value of TopDown metric events can be calculated by multiplying the METRICS (percentage) register with SLOTS fixed counter.
New in Icelake
- Do not require generic counters. This allows to collect TopDown always in addition to other events.
- Measuring TopDown per thread/process instead of only per coreLimitation
- To get accurate result and avoid reading the METRICS register multiple times, the TopDown metrics events and SLOTS event have to be in the same group.
- METRICS and SLOTS registers have to be cleared after each read by SW. That is to prevent the lose of precision and a known side effect of METRICS register.
- Cannot do sampling read SLOTS and TopDown metric eventsPlease refer SDM Vol3, 18.3.9.3 Performance Metrics for the details of TopDown metrics.
Changes since V2:
- Rebase on top of v5.3-rc1Key changes since V1:
- Remove variables forreg_idxand enabled_events[] array. Thereg_idxcan be calculated by idx in runtime.
Using existingactive_maskto replaceenabled_events.
- Choose value 47 for the fixed index of BTS.
- SupportOVF_PERF_METRICSoverflow bit in PMI handler
- Drops the caching mechanism and related variables
New mechanism is to update all active slots/metrics events for the first slots/metrics events in a group. For each group reading, it still only read the slots/perf_metrics MSR once
- Disable PMU for read of topdown events to avoid the NMI issue
- Move RDPMC support to a separate patch
- Using event=0x00,umask=0x1X for topdown metrics events
- Drop the patch which add REMOVE transaction
We can indicatex86_pmu_stop()by checking
(event && !test_bit(event->hw.idx,cpuc->active_mask)), which is a good place to save the slots/metrics MSR valueAndi Kleen (2):
perf, tools, stat: Support new per thread TopDown metrics perf, tools: Add documentation for topdown metricsKan Liang (6):
perf/x86/intel: Set correct mask for TOPDOWN.SLOTS perf/x86/intel: Basic support for metrics counters perf/x86/intel: Support hardware TopDown metrics perf/x86/intel: Support per thread RDPMC TopDown metrics perf/x86/intel: Export TopDown events for Icelake perf/x86/intel: Disable sampling read slots and topdownarch/x86/events/core.c | 35 ++- arch/x86/events/intel/core.c | 362 ++++++++++++++++++++++++- arch/x86/events/perf_event.h | 33 +++ arch/x86/include/asm/msr-index.h | 3 + arch/x86/include/asm/perf_event.h | 54 +++- include/linux/perf_event.h | 3 + tools/perf/Documentation/perf-stat.txt | 9 +- tools/perf/Documentation/topdown.txt | 223 +++++++++++++++ tools/perf/builtin-stat.c | 24 ++ tools/perf/util/stat-shadow.c | 89 ++++++ tools/perf/util/stat.c | 4 + tools/perf/util/stat.h | 8 + 12 files changed, 827 insertions(+), 20 deletions(-)create mode 100644
tools/perf/Documentation/topdown.txt