TopDown metrics support for Icelake
Subsystems (2)
v6 cover TopDown metrics support for Icelake
From: Kan Liang <[email protected]>
Changes since V5:
- Rebase on top of tip perf/core branch
commit c085fb877467 ("perf/x86/intel/lbr: Support XSAVES for arch LBR read")
- Add more cleanup codes, e.g. ("perf/x86/intel: Use switch in intel_pmu_disable/enable_event"). Use more macros to replace the magic number.
- Reorganize the patch set:
- Move all cleanup related patches to the beginning of the patch set.
- Split the TopDown metrics patch into a generic patch and a Icelake specific patch.
- Drop extra_reg. The TopDown metrics events and slots event don't support any filters.
Changes since V4:
- Add description regarding to event-code naming for fixed counters
- Fix add_nr_metric_event().
For leader event, we have to take the accepted metrics events into account.
For sibling event, it doesn't need to count accepted metrics events again.
- Remove is_first_topdown_event_in_group().
Force slots in topdown group. Only update topdown events with slots event.
- Re-use last_period and period_left for saved_metric and saved_slots.
Changes since V3:
- Separate fixed counter3 definition patch
- Separate BTS index patch
- Apply Peter's cleanup patch
- Fix the name of perf capabilities for perf METRICS
- Apply patch for mul_u64_u32_div() x86_64 implementation
- Fix unconditionally allows collecting 4 extra events
- Add patch to clean up NMI handler by naming global status bit
- Add patch to reuse event_base_rdpmc for RDPMC userspace support
Changes since V2:
- Rebase on top of v5.3-rc1
Key changes since V1:
- Remove variables for reg_idx and enabled_events[] array. The reg_idx can be calculated by idx in runtime.
Using existing active_mask to replace enabled_events.
- Choose value 47 for the fixed index of BTS.
- Support OVF_PERF_METRICS overflow bit in PMI handler
- Drops the caching mechanism and related variables
New mechanism is to update all active slots/metrics events for the first slots/metrics events in a group. For each group reading, it still only read the slots/perf_metrics MSR once
- Disable PMU for read of topdown events to avoid the NMI issue
- Move RDPMC support to a separate patch
- Using event=0x00,umask=0x1X for topdown metrics events
- Drop the patch which add REMOVE transaction
We can indicate x86_pmu_stop() by checking
(event && !test_bit(event->hw.idx, cpuc->active_mask)), which is a good place to save the slots/metrics MSR value
Icelake has support for measuring the level 1 TopDown metrics directly in hardware. This is implemented by an additional METRICS register, and a new Fixed Counter 3 that measures pipeline SLOTS.
New in Icelake
- Do not require generic counters. This allows to collect TopDown always in addition to other events.
- Measuring TopDown per thread/process instead of only per core
For the Ice Lake implementation of performance metrics, the values in PERF_METRICS MSR are derived from fixed counter 3. Software should start both registers, PERF_METRICS and fixed counter 3, from zero. Additionally, software is recommended to periodically clear both registers in order to maintain accurate measurements. The latter is required for certain scenarios that involve sampling metrics at high rates. Software should always write fixed counter 3 before write to PERF_METRICS.
IA32_PERF_GLOBAL_STATUS. OVF_PERF_METRICS[48]: If this bit is set, it indicates that some PERF_METRICS-related counter has overflowed and a PMI is triggered. Software has to synchronize, e.g. re-start, PERF_METRICS as well as fixed counter 3. Otherwise, PERF_METRICS may return invalid values.
Limitation
- To get accurate result and avoid reading the METRICS register multiple times, the TopDown metrics events and SLOTS event have to be in the same group.
- METRICS and SLOTS registers have to be cleared after each read by SW. That is to prevent the lose of precision.
- Cannot do sampling read SLOTS and TopDown metric events
Please refer SDM Vol3, 18.3.9.3 Performance Metrics for the details of TopDown metrics.
Andi Kleen (2):
perf, tools, stat: Support new per thread TopDown metrics perf, tools: Add documentation for topdown metrics
Kan Liang (12):
perf/x86: Use event_base_rdpmc for the RDPMC userspace support perf/x86/intel: Name the global status bit in NMI handler perf/x86/intel: Introduce the fourth fixed counter perf/x86/intel: Move BTS index to 47
perf/x86/intel: Fix the name of perf METRICS perf/x86/intel: Use switch in intel_pmu_disable/enable_event perf/x86/intel: Generic support for hardware TopDown metrics perf/x86: Add a macro for RDPMC offset of fixed counters perf/x86/intel: Support TopDown metrics on Ice Lake perf/x86/intel: Support per-thread RDPMC TopDown metrics perf/x86/intel: Disable sample-read the slots and metrics events perf, tools, stat: Check Topdown Metric group
arch/x86/events/core.c | 71 ++++- arch/x86/events/intel/core.c | 356 +++++++++++++++++++++++-- arch/x86/events/perf_event.h | 56 +++- arch/x86/include/asm/msr-index.h | 3 + arch/x86/include/asm/perf_event.h | 96 ++++++- include/linux/perf_event.h | 29 +- tools/perf/Documentation/perf-stat.txt | 9 +- tools/perf/Documentation/topdown.txt | 235 ++++++++++++++++ tools/perf/builtin-stat.c | 97 +++++++ tools/perf/util/stat-shadow.c | 89 +++++++ tools/perf/util/stat.c | 4 + tools/perf/util/stat.h | 8 + 12 files changed, 1002 insertions(+), 51 deletions(-) create mode 100644 tools/perf/Documentation/topdown.txt
Signature · 1 line
2.17.1
PATCH v6 1/14 perf/x86: Use event_base_rdpmc for the RDPMC userspace support
From: Kan Liang <[email protected]>
The RDPMC index is always re-calculated for the RDPMC userspace support, which is unnecessary.
The RDPMC index value is stored in the variable event_base_rdpmc for the kernel usage, which can be used for RDPMC userspace support as well.
--- arch/x86/events/core.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-)
arch/x86/events/core.c +3 -8
@@ -2208,17 +2208,12 @@ static void x86_pmu_event_unmapped(struct perf_event *event, struct mm_struct *mstatic int x86_pmu_event_idx (struct perf_event *event) { -int idx = event->hw.idx; +struct hw_perf_event *hwc = &event->hw; -if (!(event->hw.flags &PERF_X86_EVENT_RDPMC_ALLOWED )) +if (!(hwc->flags &PERF_X86_EVENT_RDPMC_ALLOWED ))return 0 ; -if (x86_pmu.num_counters_fixed && idx >=INTEL_PMC_IDX_FIXED ) { - idx -=INTEL_PMC_IDX_FIXED ; - idx |=1 <<30 ; - } - -return idx +1 ; +return hwc->event_base_rdpmc +1 ; } static ssize_t get_attr_rdpmc(struct device *cdev,
Signature · 1 line
2.17.1
PATCH v6 3/14 perf/x86/intel: Introduce the fourth fixed counter
In reply to [email protected]From: Kan Liang <[email protected]>
The fourth fixed counter, TOPDOWN.SLOTS, is introduced in Ice Lake to measure the level 1 TopDown events.
Add MSR address and macros for the new fixed counter, which will be used in a later patch.
Add comments to explain the event encoding rules for the fixed counters.
--- arch/x86/include/asm/perf_event.h | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
arch/x86/include/asm/perf_event.h +20 -3
@@ -197,12 +197,24 @@ struct x86_pmu_capability {*/ /* -* All 3 fixed-mode PMCs are configured via this single MSR: +* All the fixed-mode PMCs are configured via this single MSR: */ #define MSR_ARCH_PERFMON_FIXED_CTR_CTRL 0x38d/* -* The counts are available in three separate MSRs: +* There is no event-code assigned to the fixed-mode PMCs. +* +* For a fixed-mode PMC, which has an equivalent event on a general-purpose +* PMC, the event-code of the equivalent event is used for the fixed-mode PMC, +* e.g., Instr_Retired.Any and CPU_CLK_Unhalted.Core. +* +* For a fixed-mode PMC, which doesn't have an equivalent event, a +* pseudo-encoding is used, e.g., CPU_CLK_Unhalted.Ref and TOPDOWN.SLOTS. +* The pseudo event-code for a fixed-mode PMC must be 0x00. +* The pseudo umask-code is 0x0X. The X indicates the index of the fixed +* counter. +* +* The counts are available in separate MSRs: */ /* Instr_Retired.Any: */
@@ -213,11 +225,16 @@ struct x86_pmu_capability {#define MSR_ARCH_PERFMON_FIXED_CTR1 0x30a#define INTEL_PMC_IDX_FIXED_CPU_CYCLES (INTEL_PMC_IDX_FIXED + 1) -/* CPU_CLK_Unhalted.Ref: */ +/* CPU_CLK_Unhalted.Ref: event=0x00,umask=0x3 (pseudo-encoding) */ #define MSR_ARCH_PERFMON_FIXED_CTR2 0x30b#define INTEL_PMC_IDX_FIXED_REF_CYCLES (INTEL_PMC_IDX_FIXED + 2)#define INTEL_PMC_MSK_FIXED_REF_CYCLES (1ULL << INTEL_PMC_IDX_FIXED_REF_CYCLES) +/* TOPDOWN.SLOTS: event=0x00,umask=0x4 (pseudo-encoding) */ +#define MSR_ARCH_PERFMON_FIXED_CTR3 0x30c +#define INTEL_PMC_IDX_FIXED_SLOTS (INTEL_PMC_IDX_FIXED + 3) +#define INTEL_PMC_MSK_FIXED_SLOTS (1ULL << INTEL_PMC_IDX_FIXED_SLOTS) +/* * We model BTS tracing as another fixed-mode PMC. *
Signature · 1 line
2.17.1
PATCH v6 10/14 perf/x86/intel: Support per-thread RDPMC TopDown metrics
In reply to [email protected]From: Kan Liang <[email protected]>
Starts from Ice Lake, the TopDown metrics are directly available as fixed counters and do not require generic counters. Also, the TopDown metrics can be collected per thread. Extend the RDPMC usage to support per-thread TopDown metrics.
The RDPMC index of the PERF_METRICS will be output if RDPMC users ask for the RDPMC index of the metrics events.
To support per thread RDPMC TopDown, the metrics and slots counters have to be saved/restored during the context switching.
The last_period and period_left are not used in the counting mode. Use the fields for saved_metric and saved_slots.
--- arch/x86/events/core.c | 5 +- arch/x86/events/intel/core.c | 103 +++++++++++++++++++++++++++++------ include/linux/perf_event.h | 29 ++++++---- 3 files changed, 108 insertions(+), 29 deletions(-)
arch/x86/events/core.c +4 -1
@@ -2264,7 +2264,10 @@ static int x86_pmu_event_idx(struct perf_event *event)if (!(hwc->flags &PERF_X86_EVENT_RDPMC_ALLOWED ))return 0 ; -return hwc->event_base_rdpmc +1 ; +if (is_metric_idx (hwc->idx)) +return INTEL_PMC_FIXED_RDPMC_METRICS +1 ; +else +return hwc->event_base_rdpmc +1 ; } static ssize_t get_attr_rdpmc(struct device *cdev,
arch/x86/events/intel/core.c +85 -18
146 lines not on this page · raw message
include/linux/perf_event.h +19 -10
@@ -212,17 +212,26 @@ struct hw_perf_event {*/ u64 sample_period; -/* -* The period we started this sample with. -*/ -u64 last_period; +union { +struct {/* Sampling */ +/* +* The period we started this sample with. +*/ +u64 last_period; -/* -* However much is left of the current period; note that this is -* a full 64bit value and allows for generation of periods longer -* than hardware might allow. -*/ -local64_t period_left; +/* +* However much is left of the current period; +* note that this is a full 64bit value and +* allows for generation of periods longer +* than hardware might allow. +*/ +local64_t period_left; + }; +struct {/* Topdown events counting for context switch */ +u64 saved_metric; +u64 saved_slots; + }; + };/* * State for throttling the event, see __perf_event_overflow() and
Signature · 1 line
2.17.1
PATCH v6 9/14 perf/x86/intel: Support TopDown metrics on Ice Lake
In reply to [email protected]From: Kan Liang <[email protected]>
Ice Lake supports the hardware TopDown metrics feature, which can free up the scarce GP counters.
Update the event constraints for the metrics events. The metric counters do not exist, which are mapped to a dummy offset. The sharing between multiple users of the same metric without multiplexing is not allowed.
Implement set_topdown_event_period for Ice Lake. The values in PERF_METRICS MSR are derived from the fixed counter 3. Both registers should start from zero.
Implement update_topdown_event for Ice Lake. The metric is reported by multiplying the metric (fraction) with slots. To maintain accurate measurements, both registers are cleared for each update. The fixed counter 3 should always be cleared before the PERF_METRICS.
Implement td_attr for the new metrics events and the new slots fixed counter. Make them visible to the perf user tools.
--- arch/x86/events/intel/core.c | 124 ++++++++++++++++++++++++++++++ arch/x86/events/perf_event.h | 13 ++++ arch/x86/include/asm/msr-index.h | 2 + arch/x86/include/asm/perf_event.h | 2 + 4 files changed, 141 insertions(+)
arch/x86/events/intel/core.c +124 -0
@@ -247,6 +247,10 @@ static struct event_constraint intel_icl_event_constraints[] = {FIXED_EVENT_CONSTRAINT (0x003c ,1 ),/* CPU_CLK_UNHALTED.CORE */ FIXED_EVENT_CONSTRAINT (0x0300 ,2 ),/* CPU_CLK_UNHALTED.REF */ FIXED_EVENT_CONSTRAINT (0x0400 ,3 ),/* SLOTS */ +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_RETIRING ,0 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_BAD_SPEC ,1 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_FE_BOUND ,2 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_BE_BOUND ,3 ),INTEL_EVENT_CONSTRAINT_RANGE (0x03 ,0x0a ,0xf ),INTEL_EVENT_CONSTRAINT_RANGE (0x1f ,0x28 ,0xf ), INTEL_EVENT_CONSTRAINT(0x32, 0xf), /* SW_PREFETCH_ACCESS.* */
@@ -309,6 +313,12 @@ EVENT_ATTR_STR_HT(topdown-recovery-bubbles, td_recovery_bubbles,EVENT_ATTR_STR_HT (topdown-recovery-bubbles.scale, td_recovery_bubbles_scale,"4" ,"2" ); +EVENT_ATTR_STR (slots, slots,"event=0x00,umask=0x4" ); +EVENT_ATTR_STR (topdown-retiring, td_retiring,"event=0x00,umask=0x10" ); +EVENT_ATTR_STR (topdown-bad-spec, td_bad_spec,"event=0x00,umask=0x11" ); +EVENT_ATTR_STR (topdown-fe-bound, td_fe_bound,"event=0x00,umask=0x12" ); +EVENT_ATTR_STR (topdown-be-bound, td_be_bound,"event=0x00,umask=0x13" ); +static struct attribute *snb_events_attrs[] = {EVENT_PTR (td_slots_issued),EVENT_PTR (td_slots_retired),
@@ -2232,6 +2242,105 @@ static void intel_pmu_del_event(struct perf_event *event)intel_pmu_pebs_del (event); } +static int icl_set_topdown_event_period (struct perf_event *event) +{ +struct hw_perf_event *hwc = &event->hw; +s64 left =local64_read (&hwc->period_left); + +/* +* The values in PERF_METRICS MSR are derived from fixed counter 3. +* Software should start both registers, PERF_METRICS and fixed +* counter 3, from zero. +* Clear PERF_METRICS and Fixed counter 3 in initialization. +* After that, both MSRs will be cleared for each read. +* Don't need to clear them again. +*/ +if (left == x86_pmu.max_period) { +wrmsrl (MSR_CORE_PERF_FIXED_CTR3 ,0 ); +wrmsrl (MSR_PERF_METRICS ,0 ); +local64_set (&hwc->period_left,0 ); + } + +perf_event_update_userpage (event); + +return 0 ; +} + +static inline u64 icl_get_metrics_event_value (u64 metric,u64 slots,int idx) +{ +u32 val; + +/* +* The metric is reported as an 8bit integer fraction +* suming up to 0xff. +* slots-in-metric = (Metric / 0xff) * slots +*/ + val = (metric >> ((idx -INTEL_PMC_IDX_METRIC_BASE ) *8 )) &0xff ; +return mul_u64_u32_div (slots, val,0xff ); +} + +static void __icl_update_topdown_event (struct perf_event *event, +u64 slots,u64 metrics) +{ +int idx = event->hw.idx; +u64 delta; + +if (is_metric_idx (idx)) + delta =icl_get_metrics_event_value (metrics, slots, idx); +else + delta = slots; + +local64_add (delta, &event->count); +} + +/* +* Update all active Topdown events. +* +* The PERF_METRICS and Fixed counter 3 are read separately. The values may be +* modify by a NMI. PMU has to be disabled before calling this function. +*/ +static u64 icl_update_topdown_event (struct perf_event *event) +{ +struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +struct perf_event *other; +u64 slots, metrics; +int idx; + +/* read Fixed counter 3 */ +rdpmcl ((3 |INTEL_PMC_FIXED_RDPMC_BASE ), slots); +if (!slots) +return 0 ; + +/* read PERF_METRICS */ +rdpmcl (INTEL_PMC_FIXED_RDPMC_METRICS , metrics); + +for_each_set_bit (idx, cpuc->active_mask,INTEL_PMC_IDX_TD_BE_BOUND +1 ) { +if (!is_topdown_idx (idx)) +continue ; + other = cpuc->events[idx]; +__icl_update_topdown_event (other, slots, metrics); + } + +/* +* Check and update this event, which may have been cleared +* in active_mask e.g. x86_pmu_stop() +*/ +if (event && !test_bit (event->hw.idx, cpuc->active_mask)) +__icl_update_topdown_event (event, slots, metrics); + +/* +* Software is recommended to periodically clear both registers +* in order to maintain accurate measurements, which is required for +* certain scenarios that involve sampling metrics at high rates. +* Software should always write fixed counter 3 before write to +* PERF_METRICS. +*/ +wrmsrl (MSR_CORE_PERF_FIXED_CTR3 ,0 ); +wrmsrl (MSR_PERF_METRICS ,0 ); + +return slots; +} +static void intel_pmu_read_topdown_event (struct perf_event *event) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);
@@ -4461,6 +4570,15 @@ static struct attribute *icl_events_attrs[] = { NULL, }; +static struct attribute *icl_td_events_attrs[] = { +EVENT_PTR (slots), +EVENT_PTR (td_retiring), +EVENT_PTR (td_bad_spec), +EVENT_PTR (td_fe_bound), +EVENT_PTR (td_be_bound), +NULL , +}; +static struct attribute *icl_tsx_events_attrs[] = {EVENT_PTR (tx_start),EVENT_PTR (tx_abort),
@@ -5245,10 +5363,13 @@ __init int intel_pmu_init(void) hsw_format_attr : nhm_format_attr; extra_skl_attr = skl_format_attr; mem_attr = icl_events_attrs; + td_attr = icl_td_events_attrs; tsx_attr = icl_tsx_events_attrs; x86_pmu.rtm_abort_event = X86_CONFIG(.event=0xca, .umask=0x02); x86_pmu.lbr_pt_coexist = true;intel_pmu_pebs_data_source_skl (pmem); + x86_pmu.update_topdown_event = icl_update_topdown_event; + x86_pmu.set_topdown_event_period = icl_set_topdown_event_period;pr_cont ("Icelake events, " ); name ="icelake" ;break ;
@@ -5368,6 +5489,9 @@ __init int intel_pmu_init(void)if (x86_pmu.counter_freezing) x86_pmu.handle_irq = intel_pmu_handle_irq_v4; +if (x86_pmu.intel_cap.perf_metrics) + x86_pmu.intel_ctrl |=1ULL <<GLOBAL_CTRL_EN_PERF_METRICS ; +return 0 ; }
arch/x86/events/perf_event.h +13 -0
@@ -406,6 +406,19 @@ struct cpu_hw_events {#define FIXED_EVENT_CONSTRAINT (c, n) \ EVENT_CONSTRAINT(c, (1ULL << (32+n)), FIXED_EVENT_FLAGS) +/* +* The special metric counters do not actually exist. They are calculated from +* the combination of the FxCtr3 + MSR_PERF_METRICS. +* +* The special metric counters are mapped to a dummy offset for the scheduler. +* The sharing between multiple users of the same metric without multiplexing +* is not allowed, even though the hardware supports that in principle. +*/ + +#define METRIC_EVENT_CONSTRAINT (c, n) \ + EVENT_CONSTRAINT(c, (1ULL << (INTEL_PMC_IDX_METRIC_BASE + n)), \ + INTEL_ARCH_EVENT_MASK) +/* * Constraint on the Event code + UMask */
arch/x86/include/asm/msr-index.h +2 -0
@@ -859,6 +859,8 @@#define MSR_CORE_PERF_GLOBAL_CTRL 0x0000038f#define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x00000390 +#define MSR_PERF_METRICS 0x00000329 +/* PERF_GLOBAL_OVF_CTL bits */ #define MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT 55#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI (1ULL << MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT)
arch/x86/include/asm/perf_event.h +2 -0
@@ -198,6 +198,7 @@ struct x86_pmu_capability {/* RDPMC offset for Fixed PMCs */ #define INTEL_PMC_FIXED_RDPMC_BASE (1 << 30) +#define INTEL_PMC_FIXED_RDPMC_METRICS (1 << 29)/* * All the fixed-mode PMCs are configured via this single MSR:
@@ -304,6 +305,7 @@ static inline bool is_topdown_idx(int idx)#define GLOBAL_STATUS_TRACE_TOPAPMI BIT_ULL(GLOBAL_STATUS_TRACE_TOPAPMI_BIT)#define GLOBAL_STATUS_PERF_METRICS_OVF_BIT 48 +#define GLOBAL_CTRL_EN_PERF_METRICS 48/* * We model guest LBR event tracing as another fixed-mode PMC like BTS. *
Signature · 1 line
2.17.1
PATCH v6 14/14 perf, tools: Add documentation for topdown metrics
In reply to [email protected]From: Andi Kleen <[email protected]>
Add some documentation how to use the topdown metrics in ring 3.
--- tools/perf/Documentation/topdown.txt | 235 +++++++++++++++++++++++++++ 1 file changed, 235 insertions(+) create mode 100644 tools/perf/Documentation/topdown.txt
tools/perf/Documentation/topdown.txt new file +235 -0
@@ -0,0 +1,235 @@ +Using TopDown metrics in user space +----------------------------------- + +Intel CPUs (since Sandy Bridge and Silvermont) support a TopDown +methology to break down CPU pipeline execution into 4 bottlenecks: +frontend bound, backend bound, bad speculation, retiring. + +For more details on Topdown see [1][5] + +Traditionally this was implemented by events in generic counters +and specific formulas to compute the bottlenecks. + +perf stat --topdown implements this. + +Full Top Down includes more levels that can break down the +bottlenecks further. This is not directly implemented in perf, +but available in other tools that can run on top of perf, +such as toplev[2] or vtune[3] + +New Topdown features in Ice Lake +=============================== + +With Ice Lake CPUs the TopDown metrics are directly available as +fixed counters and do not require generic counters. This allows +to collect TopDown always in addition to other events. + +% perf stat -a --topdown -I1000 +# time counts unit events + 1.000854735 20,097,158,100 slots + 1.000854735 79,327,616 topdown-retiring # 0.4% retiring + 1.000854735 157,932,715 topdown-bad-spec # 0.8% bad speculation + 1.000854735 81,610,855 topdown-fe-bound # 0.4% frontend bound + 1.000854735 19,778,286,903 topdown-be-bound # 98.4% backend bound + 2.003623823 20,010,908,365 slots + 2.003623823 79,905,340 topdown-retiring # 0.4% retiring + 2.003623823 158,405,024 topdown-bad-spec # 0.8% bad speculation + 2.003623823 87,980,097 topdown-fe-bound # 0.4% frontend bound + 2.003623823 19,684,617,888 topdown-be-bound # 98.4% backend bound + 3.005828889 20,062,101,220 slots + 3.005828889 80,077,032 topdown-retiring # 0.4% retiring + 3.005828889 158,682,921 topdown-bad-spec # 0.8% bad speculation + 3.005828889 86,579,604 topdown-fe-bound # 0.4% frontend bound + 3.005828889 19,736,761,649 topdown-be-bound # 98.4% backend bound +... + +This also enables measuring TopDown per thread/process instead +of only per core. + +Using TopDown through RDPMC in applications on Ice Lake +====================================================== + +For more fine grained measurements it can be useful to +access the new directly from user space. This is more complicated, +but drastically lowers overhead. + +On Ice Lake, there is a new fixed counter 3: SLOTS, which reports +"pipeline SLOTS" (cycles multiplied by core issue width) and a +metric register that reports slots ratios for the different bottleneck +categories. + +The metrics counter is CPU model specific and is not be available +on older CPUs. + +Example code +============ + +Library functions to do the functionality described below +is also available in libjevents [4] + +The application opens a perf_event file descriptor +and sets up fixed counter 3 (SLOTS) to start and +allow user programs to read the performance counters. + +Fixed counter 3 is mapped to a pseudo event event=0x00, umask=04, +so the perf_event_attr structure should be initialized with +{ .config = 0x0400, .type = PERF_TYPE_RAW } + +#include <linux/perf_event.h> +#include <sys/syscall.h> +#include <unistd.h> + +/* Provide own perf_event_open stub because glibc doesn't */ +__attribute__((weak)) +int perf_event_open(struct perf_event_attr *attr, pid_t pid, + int cpu, int group_fd, unsigned long flags) +{ + return syscall(__NR_perf_event_open, attr, pid, cpu, group_fd, flags); +} + +/* open slots counter file descriptor for current task */ +struct perf_event_attr slots = { + .type = PERF_TYPE_RAW, + .size = sizeof(struct perf_event_attr), + .config = 0x400, + .exclude_kernel = 1, +}; + +int fd = perf_event_open(&slots, 0, -1, -1, 0); +if (fd < 0) + ... error ... + +The RDPMC instruction (or _rdpmc compiler intrinsic) can now be used +to read slots and the topdown metrics at different points of the program: + +#include <stdint.h> +#include <x86intrin.h> + +#define RDPMC_FIXED (1 << 30) /* return fixed counters */ +#define RDPMC_METRIC (1 << 29) /* return metric counters */ + +#define FIXED_COUNTER_SLOTS 3 +#define METRIC_COUNTER_TOPDOWN_L1 0 + +static inline uint64_t read_slots(void) +{ + return _rdpmc(RDPMC_FIXED | FIXED_COUNTER_SLOTS); +} + +static inline uint64_t read_metrics(void) +{ + return _rdpmc(RDPMC_METRIC | METRIC_COUNTER_TOPDOWN_L1); +} + +Then the program can be instrumented to read these metrics at different +points. + +It's not a good idea to do this with too short code regions, +as the parallelism and overlap in the CPU program execution will +cause too much measurement inaccuracy. For example instrumenting +individual basic blocks is definitely too fine grained. + +Decoding metrics values +======================= + +The value reported by read_metrics() contains four 8 bit fields +that represent a scaled ratio that represent the Level 1 bottleneck. +All four fields add up to 0xff (= 100%) + +The binary ratios in the metric value can be converted to float ratios: + +#define GET_METRIC(m, i) (((m) >> (i*8)) & 0xff) + +#define TOPDOWN_RETIRING(val) ((float)GET_METRIC(val, 0) / 0xff) +#define TOPDOWN_BAD_SPEC(val) ((float)GET_METRIC(val, 1) / 0xff) +#define TOPDOWN_FE_BOUND(val) ((float)GET_METRIC(val, 2) / 0xff) +#define TOPDOWN_BE_BOUND(val) ((float)GET_METRIC(val, 3) / 0xff) + +and then converted to percent for printing. + +The ratios in the metric accumulate for the time when the counter +is enabled. For measuring programs it is often useful to measure +specific sections. For this it is needed to deltas on metrics. + +This can be done by scaling the metrics with the slots counter +read at the same time. + +Then it's possible to take deltas of these slots counts +measured at different points, and determine the metrics +for that time period. + + slots_a = read_slots(); + metric_a = read_metrics(); + + ... larger code region ... + + slots_b = read_slots() + metric_b = read_metrics() + + # compute scaled metrics for measurement a + retiring_slots_a = GET_METRIC(metric_a, 0) * slots_a + bad_spec_slots_a = GET_METRIC(metric_a, 1) * slots_a + fe_bound_slots_a = GET_METRIC(metric_a, 2) * slots_a + be_bound_slots_a = GET_METRIC(metric_a, 3) * slots_a + + # compute delta scaled metrics between b and a + retiring_slots = GET_METRIC(metric_b, 0) * slots_b - retiring_slots_a + bad_spec_slots = GET_METRIC(metric_b, 1) * slots_b - bad_spec_slots_a + fe_bound_slots = GET_METRIC(metric_b, 2) * slots_b - fe_bound_slots_a + be_bound_slots = GET_METRIC(metric_b, 3) * slots_b - be_bound_slots_a + +Later the individual ratios for the measurement period can be recreated +from these counts. + + slots_delta = slots_b - slots_a + retiring_ratio = (float)retiring_slots / slots_delta + bad_spec_ratio = (float)bad_spec_slots / slots_delta + fe_bound_ratio = (float)fe_bound_slots / slots_delta + be_bound_ratio = (float)be_bound_slots / slota_delta + + printf("Retiring %.2f%% Bad Speculation %.2f%% FE Bound %.2f%% BE Bound %.2f%%\n", + retiring_ratio * 100., + bad_spec_ratio * 100., + fe_bound_ratio * 100., + be_bound_ratio * 100.); + +Resetting metrics counters +========================== + +Since the individual metrics are only 8bit they lose precision for +short regions over time because the number of cycles covered by each +fraction bit shrinks. So the counters need to be reset regularly. + +When using the kernel perf API the kernel resets on every read. +So as long as the reading is at reasonable intervals (every few +seconds) the precision is good. + +When using perf stat it is recommended to always use the -I option, +with no longer interval than a few seconds + + perf stat -I 1000 --topdown ... + +For user programs using RDPMC directly the counter can +be reset explicitly using ioctl: + + ioctl(perf_fd, PERF_EVENT_IOC_RESET, 0); + +This "opens" a new measurement period. + +A program using RDPMC for TopDown should schedule such a reset +regularly, as in every few seconds. + +Limits on Ice Lake +================== + +All the TopDown events must be in a group with SLOTS events. + +There is no sampling support for TopDown events. +Sampling read SLOTS and TopDown events is forbidden. +For example, perf record -e '{slots, topdown-retiring}:S' + +[1] https://software.intel.com/en-us/top-down-microarchitecture-analysis-method-win +[2] https://github.com/andikleen/pmu-tools/wiki/toplev-manual +[3] https://software.intel.com/en-us/intel-vtune-amplifier-xe +[4] https://github.com/andikleen/pmu-tools/tree/master/jevents +[5] https://sites.google.com/site/analysismethods/yasin-pubs
Signature · 1 line
2.17.1
PATCH v6 13/14 perf, tools, stat: Check Topdown Metric group
In reply to [email protected]From: Kan Liang <[email protected]>
The slots event is required in a Topdown Metric group.
Add a check to examine the Topdown Metric group. Error out if there is no slots event detected.
Only check the group on the platform which using topdown_metric_attrs, e.g. Ice Lake.
--- tools/perf/builtin-stat.c | 72 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+)
tools/perf/builtin-stat.c +72 -0
@@ -1441,6 +1441,72 @@ static int topdown_filter_events(const char **attr, char **str, bool use_group)return 0 ; } +/* Event encoding for Topdown Metric events */ +#define TOPDOWN_SLOTS 0x0400 +#define TOPDOWN_RETIRE 0x1000 +#define TOPDOWN_BAD_SPEC 0x1100 +#define TOPDOWN_FE_BOUND 0x1200 +#define TOPDOWN_BE_BOUND 0x1300 + +static bool is_topdown_metric_event (struct evsel *counter) +{ +if (!counter->pmu_name) +return false; + +if (strcmp (counter->pmu_name,"cpu" )) +return false; + +if ((counter->core.attr.config ==TOPDOWN_RETIRE ) || + (counter->core.attr.config ==TOPDOWN_BAD_SPEC ) || + (counter->core.attr.config ==TOPDOWN_FE_BOUND ) || + (counter->core.attr.config ==TOPDOWN_BE_BOUND )) +return true; + +return false; +} + +static bool is_topdown_slots_event (struct evsel *counter) +{ +if (!counter->pmu_name) +return false; + +if (strcmp (counter->pmu_name,"cpu" )) +return false; + +if (counter->core.attr.config ==TOPDOWN_SLOTS ) +return true; + +return false; +} + +static bool topdown_check_group_member (void ) +{ +struct evsel *counter, *leader, *member; +bool has_slots; + +if (!pmu_have_event ("cpu" , topdown_metric_attrs[0 ])) +return true; + +evlist__for_each_entry (evsel_list, counter) { +if (!is_topdown_metric_event (counter)) +continue ; + + leader = counter->leader; + has_slots = false; + +for_each_group_evsel (member, leader) { +if (is_topdown_slots_event (member)) + has_slots = true; + counter = member; + } + +if (!has_slots) +return false; + } + +return true; +} +__weak bool arch_topdown_check_group (bool *warn) { *warn = false;
@@ -2024,6 +2090,12 @@ int cmd_stat(int argc, const char **argv) (const char **) stat_usage, PARSE_OPT_STOP_AT_NON_OPTION);perf_stat__collect_metric_expr (evsel_list); + +if (!topdown_check_group_member ()) { +fprintf (stderr,"Topdown group must include slots event\n" ); + goto out; + } +perf_stat__init_shadow_stats ();if (stat_config.csv_sep) {
Signature · 1 line
2.17.1
PATCH v6 7/14 perf/x86/intel: Generic support for hardware TopDown metrics
In reply to [email protected]From: Kan Liang <[email protected]>
Intro
=====
The TopDown Microarchitecture Analysis (TMA) Method is a structured analysis methodology to identify critical performance bottlenecks in out-of-order processors. Current perf has supported the method.
The method works well, but there is one problem. To collect the TopDown events, several GP counters have to be used. If a user wants to collect other events at the same time, the multiplexing probably be triggered, which impacts the accuracy.
To free up the scarce GP counters, the hardware TopDown metrics feature is introduced from Ice Lake. The hardware implements an additional "metrics" register and a new Fixed Counter 3 that measures pipeline "slots". The TopDown events can be calculated from them instead.
Events
======
The level 1 TopDown has four metrics. There is no event-code assigned to the TopDown metrics. Four metric events are exported as separate perf events, which map to the internal "metrics" counter register. Those events do not exist in hardware, but can be allocated by the scheduler.
For the event mapping, a special 0x00 event code is used, which is reserved for fake events. The metric events start from umask 0x10.
When setting up the metric events, they point to the Fixed Counter 3. They have to be specially handled.
- Add the update_topdown_event() callback to read the additional metrics MSR and generate the metrics.
- Add the set_topdown_event_period() callback to initialize metrics MSR and the fixed counter 3.
- Add a variable n_metric_event to track the number of the accepted metrics events. The sharing between multiple users of the same metric without multiplexing is not allowed.
- Only enable/disable the fixed counter 3 when there are no other active TopDown events, which avoid the unnecessary writing of the fixed control register.
- Disable the PMU when reading the metrics event. The metrics MSR and the fixed counter 3 are read separately. The values may be modified by an NMI.
All four metric events don't support sampling. Since they will be handled specially for event update, a flag PERF_X86_EVENT_TOPDOWN is introduced to indicate this case.
The slots event can support both sampling and counting. For counting, the flag is also applied.
For sampling, it will be handled normally as other normal events.
Groups
======
The slots event is required in a Topdown group.
To avoid reading the METRICS register multiple times, the metrics and slots value can only be updated by slots event in a group. All active slots and metrics events will be updated one time.
NMI
======
The METRICS related register may be overflow. The bit 48 of the STATUS register will be set. If so, PERF_METRICS and Fixed counter 3 are required to be reset. The patch also update all active slots and metrics events in the NMI handler.
The update_topdown_event() has to read two registers separately. The values may be modified by an NMI. PMU has to be disabled before calling the function.
RDPMC
======
RDPMC is temporarily disabled. A later patch will enable it.
--- arch/x86/events/core.c | 50 ++++++++++++++ arch/x86/events/intel/core.c | 105 ++++++++++++++++++++++++++++-- arch/x86/events/perf_event.h | 39 ++++++++++- arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/perf_event.h | 46 +++++++++++++ 5 files changed, 235 insertions(+), 6 deletions(-)
arch/x86/events/core.c +50 -0
@@ -76,6 +76,9 @@ u64 x86_perf_event_update(struct perf_event *event)if (unlikely (!hwc->event_base))return 0 ; +if (unlikely (is_topdown_count (event)) && x86_pmu.update_topdown_event) +return x86_pmu.update_topdown_event (event); +/* * Careful: an NMI might modify the previous event value. *
@@ -1031,6 +1034,35 @@ int x86_schedule_events(struct cpu_hw_events *cpuc, int n, int *assign)return unsched ? -EINVAL :0 ; } +static int add_nr_metric_event (struct cpu_hw_events *cpuc, +struct perf_event *event, +int *max_count,bool sibling) +{ +/* The TopDown metrics events cannot be shared. */ +if (is_metric_event (event) && + (++cpuc->n_metric_event >INTEL_TD_METRIC_NUM )) { + cpuc->n_metric_event--; +return -EINVAL ; + } + +/* +* Take the accepted metrics events into account for leader event. +*/ +if (!sibling) + *max_count += cpuc->n_metric_event; +else if (is_metric_event (event)) + (*max_count)++; + +return 0 ; +} + +static void del_nr_metric_event (struct cpu_hw_events *cpuc, +struct perf_event *event) +{ +if (is_metric_event (event)) + cpuc->n_metric_event--; +} +/* * dogrp: true if must collect siblings events (group) * returns total number of events and error code
@@ -1066,6 +1098,10 @@ static int collect_events(struct cpu_hw_events *cpuc, struct perf_event *leader, cpuc->pebs_output =is_pebs_pt (leader) +1 ; } +if (x86_pmu.intel_cap.perf_metrics && +add_nr_metric_event (cpuc, leader, &max_count, false)) +return -EINVAL ; +if (is_x86_event (leader)) {if (n >= max_count)return -EINVAL ;
@@ -1082,6 +1118,10 @@ static int collect_events(struct cpu_hw_events *cpuc, struct perf_event *leader, event->state <= PERF_EVENT_STATE_OFF)continue ; +if (x86_pmu.intel_cap.perf_metrics && +add_nr_metric_event (cpuc, event, &max_count, true)) +return -EINVAL ; +if (n >= max_count)return -EINVAL ;
@@ -1110,6 +1150,10 @@ static inline void x86_assign_hw_event(struct perf_event *event, hwc->event_base =0 ;break ; + case INTEL_PMC_IDX_METRIC_BASE ... INTEL_PMC_IDX_METRIC_BASE + 3: +/* All the metric events are mapped onto the fixed counter 3. */ + idx =INTEL_PMC_IDX_FIXED_SLOTS ; +/* fall through */ case INTEL_PMC_IDX_FIXED ... INTEL_PMC_IDX_FIXED_BTS-1: hwc->config_base =MSR_ARCH_PERFMON_FIXED_CTR_CTRL ; hwc->event_base = MSR_ARCH_PERFMON_FIXED_CTR0 +
@@ -1245,6 +1289,10 @@ int x86_perf_event_set_period(struct perf_event *event)if (unlikely (!hwc->event_base))return 0 ; +if (unlikely (is_topdown_count (event)) && + x86_pmu.set_topdown_event_period) +return x86_pmu.set_topdown_event_period (event); +/* * If we are way outside a reasonable range then just skip forward: */
@@ -1529,6 +1577,8 @@ static void x86_pmu_del(struct perf_event *event, int flags) } cpuc->event_constraint[i-1 ] =NULL ; --cpuc->n_events; +if (x86_pmu.intel_cap.perf_metrics) +del_nr_metric_event (cpuc, event);perf_event_update_userpage (event);
arch/x86/events/intel/core.c +100 -5
167 lines not on this page · raw message
arch/x86/events/perf_event.h +37 -0
@@ -79,6 +79,31 @@ static inline bool constraint_match(struct event_constraint *c, u64 ecode)#define PERF_X86_EVENT_PEBS_VIA_PT 0x0800/* use PT buffer for PEBS */ #define PERF_X86_EVENT_PAIR 0x1000/* Large Increment per Cycle */ #define PERF_X86_EVENT_LBR_SELECT 0x2000/* Save/Restore MSR_LBR_SELECT */ +#define PERF_X86_EVENT_TOPDOWN 0x4000/* Count Topdown slots/metrics events */ + +static inline bool is_topdown_count (struct perf_event *event) +{ +return event->hw.flags &PERF_X86_EVENT_TOPDOWN ; +} + +static inline bool is_metric_event (struct perf_event *event) +{ +u64 config = event->attr.config; + +return ((config &ARCH_PERFMON_EVENTSEL_EVENT ) ==0 ) && + ((config &INTEL_ARCH_EVENT_MASK ) >=INTEL_TD_METRIC_RETIRING ) && + ((config &INTEL_ARCH_EVENT_MASK ) <=INTEL_TD_METRIC_MAX ); +} + +static inline bool is_slots_event (struct perf_event *event) +{ +return (event->attr.config &INTEL_ARCH_EVENT_MASK ) ==INTEL_TD_SLOTS ; +} + +static inline bool is_topdown_event (struct perf_event *event) +{ +return is_metric_event (event) ||is_slots_event (event); +}struct amd_nb {int nb_id;/* NorthBridge id */
@@ -284,6 +309,12 @@ struct cpu_hw_events {*/ u64 tfa_shadow; +/* +* Perf Metrics +*/ +/* number of accepted metrics events */ +int n_metric_event; +/* * AMD specific bits */
@@ -726,6 +757,12 @@ struct x86_pmu {*/ atomic_t lbr_exclusive[x86_lbr_exclusive_max]; +/* +* Intel perf metrics +*/ +u64 (*update_topdown_event)(struct perf_event *event); +int (*set_topdown_event_period)(struct perf_event *event); +/* * perf task context (i.e. struct perf_event_context::task_ctx_data) * switch helper to bridge calls from perf/core to perf/x86.
arch/x86/include/asm/msr-index.h +1 -0
@@ -853,6 +853,7 @@#define MSR_CORE_PERF_FIXED_CTR0 0x00000309#define MSR_CORE_PERF_FIXED_CTR1 0x0000030a#define MSR_CORE_PERF_FIXED_CTR2 0x0000030b +#define MSR_CORE_PERF_FIXED_CTR3 0x0000030c#define MSR_CORE_PERF_FIXED_CTR_CTRL 0x0000038d#define MSR_CORE_PERF_GLOBAL_STATUS 0x0000038e#define MSR_CORE_PERF_GLOBAL_CTRL 0x0000038f
arch/x86/include/asm/perf_event.h +46 -0
58 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v6 12/14 perf, tools, stat: Support new per thread TopDown metrics
In reply to [email protected]From: Andi Kleen <[email protected]>
Icelake has support for reporting per thread TopDown metrics. These are reported differently than the previous TopDown support, each metric is standalone, but scaled to pipeline "slots". We don't need to do anything special for HyperThreading anymore. Teach perf stat --topdown to handle these new metrics and print them in the same way as the previous TopDown metrics. The restrictions of only being able to report information per core is gone.
--- tools/perf/Documentation/perf-stat.txt | 9 ++- tools/perf/builtin-stat.c | 25 ++++++++ tools/perf/util/stat-shadow.c | 89 ++++++++++++++++++++++++++ tools/perf/util/stat.c | 4 ++ tools/perf/util/stat.h | 8 +++ 5 files changed, 133 insertions(+), 2 deletions(-)
tools/perf/Documentation/perf-stat.txt +7 -2
@@ -317,8 +317,13 @@ if the workload is actually bound by the CPU and not by something else. For best results it is usually a good idea to use it with interval mode like -I 1000, as the bottleneck of workloads can change often. -The top down metrics are collected per core instead of per -CPU thread. Per core mode is automatically enabled +This enables --metric-only, unless overridden with --no-metric-only. + +The following restrictions only apply to older Intel CPUs and Atom, +on newer CPUs (IceLake and later) TopDown can be collected for any thread: + +The top down metrics are collected per core instead of per CPU thread. +Per core mode is automatically enabled and -a (global monitoring) is needed, requiring root rights or perf.perf_event_paranoid=-1.
tools/perf/builtin-stat.c +25 -0
@@ -128,6 +128,15 @@ static const char * topdown_attrs[] = { NULL, }; +static const char *topdown_metric_attrs[] = { +"slots" , +"topdown-retiring" , +"topdown-bad-spec" , +"topdown-fe-bound" , +"topdown-be-bound" , +NULL , +}; +static const char *smi_cost_attrs = {"{" "msr/aperf/,"
@@ -1638,6 +1647,21 @@ static int add_default_attributes(void)char *str =NULL ;bool warn = false; +if (topdown_filter_events (topdown_metric_attrs, &str,1 ) <0 ) { +pr_err ("Out of memory\n" ); +return -1 ; + } +if (topdown_metric_attrs[0 ] && str) { +if (!stat_config.interval) { +fprintf (stat_config.output, +"Topdown accuracy may decreases when measuring long period.\n" +"Please print the result regularly, e.g. -I1000\n" ); + } + goto setup_metrics; + } + + str =NULL ; +if (stat_config.aggr_mode !=AGGR_GLOBAL && stat_config.aggr_mode !=AGGR_CORE ) {pr_err ("top down event configuration requires --per-core mode\n" );
@@ -1659,6 +1683,7 @@ static int add_default_attributes(void)if (topdown_attrs[0 ] && str) {if (warn)arch_topdown_group_warn (); +setup_metrics: err =parse_events (evsel_list, str, &errinfo);if (err) { fprintf(stderr,
tools/perf/util/stat-shadow.c +89 -0
107 lines not on this page · raw message
tools/perf/util/stat.c +4 -0
@@ -95,6 +95,10 @@ static const char *id_str[PERF_STAT_EVSEL_ID__MAX] = {ID (TOPDOWN_SLOTS_RETIRED , topdown-slots-retired),ID (TOPDOWN_FETCH_BUBBLES , topdown-fetch-bubbles),ID (TOPDOWN_RECOVERY_BUBBLES , topdown-recovery-bubbles), +ID (TOPDOWN_RETIRING , topdown-retiring), +ID (TOPDOWN_BAD_SPEC , topdown-bad-spec), +ID (TOPDOWN_FE_BOUND , topdown-fe-bound), +ID (TOPDOWN_BE_BOUND , topdown-be-bound), ID(SMI_NUM, msr/smi/), ID(APERF, msr/aperf/), };
tools/perf/util/stat.h +8 -0
@@ -28,6 +28,10 @@ enum perf_stat_evsel_id {PERF_STAT_EVSEL_ID__TOPDOWN_SLOTS_RETIRED ,PERF_STAT_EVSEL_ID__TOPDOWN_FETCH_BUBBLES ,PERF_STAT_EVSEL_ID__TOPDOWN_RECOVERY_BUBBLES , +PERF_STAT_EVSEL_ID__TOPDOWN_RETIRING , +PERF_STAT_EVSEL_ID__TOPDOWN_BAD_SPEC , +PERF_STAT_EVSEL_ID__TOPDOWN_FE_BOUND , +PERF_STAT_EVSEL_ID__TOPDOWN_BE_BOUND ,PERF_STAT_EVSEL_ID__SMI_NUM ,PERF_STAT_EVSEL_ID__APERF , PERF_STAT_EVSEL_ID__MAX,
@@ -82,6 +86,10 @@ enum stat_type {STAT_TOPDOWN_SLOTS_RETIRED ,STAT_TOPDOWN_FETCH_BUBBLES ,STAT_TOPDOWN_RECOVERY_BUBBLES , +STAT_TOPDOWN_RETIRING , +STAT_TOPDOWN_BAD_SPEC , +STAT_TOPDOWN_FE_BOUND , +STAT_TOPDOWN_BE_BOUND ,STAT_SMI_NUM ,STAT_APERF ,STAT_MAX
Signature · 1 line
2.17.1
PATCH v6 11/14 perf/x86/intel: Disable sample-read the slots and metrics events
In reply to [email protected]From: Kan Liang <[email protected]>
Users fail to sample-read the slots and metrics events, e.g., perf record -e '{slots, topdown-retiring}:S'.
When reading the metrics event, the fixed counter 3 (slots) has to be reset, which impacts the sampling of the slots event.
Add a specific validate_group() support to reject the case and error out for Ice Lake.
An alternative fix may unconditionally disable slots sampling, but it's not a decent fix. Users may want to only sample the slot events without the topdown metrics events.
--- arch/x86/events/core.c | 4 ++++ arch/x86/events/intel/core.c | 20 ++++++++++++++++++++ arch/x86/events/perf_event.h | 2 ++ 3 files changed, 26 insertions(+)
arch/x86/events/core.c +4 -0
@@ -2176,7 +2176,11 @@ static int validate_group(struct perf_event *event) fake_cpuc->n_events =0 ; ret = x86_pmu.schedule_events (fake_cpuc, n,NULL ); +if (ret) + goto out; +if (x86_pmu.validate_group) + ret = x86_pmu.validate_group (fake_cpuc, n); out:free_fake_cpuc (fake_cpuc);return ret;
arch/x86/events/intel/core.c +20 -0
@@ -4576,6 +4576,25 @@ static __init void intel_ht_bug(void) x86_pmu.stop_scheduling = intel_stop_scheduling; } +static int icl_validate_group (struct cpu_hw_events *cpuc,int n) +{ +bool has_sampling_slots = false, has_metrics = false; +struct perf_event *e; +int i; + +for (i =0 ; i < n; i++) { + e = cpuc->event_list[i]; +if (is_slots_event (e) &&is_sampling_event (e)) + has_sampling_slots = true; + +if (is_metric_event (e)) + has_metrics = true; + } +if (unlikely (has_sampling_slots && has_metrics)) +return -EINVAL ; +return 0 ; +} +EVENT_ATTR_STR (mem-loads, mem_ld_hsw,"event=0xcd,umask=0x1,ldlat=3" );EVENT_ATTR_STR (mem-stores, mem_st_hsw,"event=0xd0,umask=0x82" )
@@ -5437,6 +5456,7 @@ __init int intel_pmu_init(void)intel_pmu_pebs_data_source_skl (pmem); x86_pmu.update_topdown_event = icl_update_topdown_event; x86_pmu.set_topdown_event_period = icl_set_topdown_event_period; + x86_pmu.validate_group = icl_validate_group;pr_cont ("Icelake events, " ); name ="icelake" ;break ;
arch/x86/events/perf_event.h +2 -0
@@ -677,6 +677,8 @@ struct x86_pmu {int perfctr_second_write;u64 (*limit_period)(struct perf_event *event,u64 l); +int (*validate_group)(struct cpu_hw_events *cpuc,int n); +/* PMI handler bits */ unsigned int late_ack :1, enabled_ack :1,
Signature · 1 line
2.17.1
PATCH v6 4/14 perf/x86/intel: Move BTS index to 47
In reply to [email protected]From: Kan Liang <[email protected]>
The bit 48 in the PERF_GLOBAL_STATUS is used to indicate the overflow status of the PERF_METRICS counters.
Move the BTS index to the bit 47.
--- arch/x86/include/asm/perf_event.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/include/asm/perf_event.h +2 -2
@@ -238,11 +238,11 @@ struct x86_pmu_capability {/* * We model BTS tracing as another fixed-mode PMC. * -* We choose a value in the middle of the fixed event range, since lower +* We choose the value 47 for the fixed index of BTS, since lower * values are used by actual fixed events and higher values are used * to indicate other overflow conditions in the PERF_GLOBAL_STATUS msr. */ -#define INTEL_PMC_IDX_FIXED_BTS (INTEL_PMC_IDX_FIXED + 16) +#define INTEL_PMC_IDX_FIXED_BTS (INTEL_PMC_IDX_FIXED + 15)#define GLOBAL_STATUS_COND_CHG BIT_ULL(63)#define GLOBAL_STATUS_BUFFER_OVF_BIT 62
Signature · 1 line
2.17.1
PATCH v6 8/14 perf/x86: Add a macro for RDPMC offset of fixed counters
In reply to [email protected]From: Kan Liang <[email protected]>
The RDPMC base offset of fixed counters is hard-code. Use a meaningful name to replace the magic number to improve the readability of the code.
--- arch/x86/events/core.c | 3 ++- arch/x86/include/asm/perf_event.h | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-)
arch/x86/events/core.c +2 -1
@@ -1158,7 +1158,8 @@ static inline void x86_assign_hw_event(struct perf_event *event, hwc->config_base =MSR_ARCH_PERFMON_FIXED_CTR_CTRL ; hwc->event_base =MSR_ARCH_PERFMON_FIXED_CTR0 + (idx -INTEL_PMC_IDX_FIXED ); - hwc->event_base_rdpmc = (idx -INTEL_PMC_IDX_FIXED ) |1 <<30 ; + hwc->event_base_rdpmc = (idx -INTEL_PMC_IDX_FIXED ) | +INTEL_PMC_FIXED_RDPMC_BASE ;break ;default :
arch/x86/include/asm/perf_event.h +3 -0
@@ -196,6 +196,9 @@ struct x86_pmu_capability {* Fixed-purpose performance events: */ +/* RDPMC offset for Fixed PMCs */ +#define INTEL_PMC_FIXED_RDPMC_BASE (1 << 30) +/* * All the fixed-mode PMCs are configured via this single MSR: */
Signature · 1 line
2.17.1
PATCH v6 6/14 perf/x86/intel: Use switch in intel_pmu_disable/enable_event
In reply to [email protected]From: Kan Liang <[email protected]>
Many items are checked in the intel_pmu_disable/enable_event. More items will be added later, e.g. perf metrics events.
Use switch, which is more efficient, to replace the if-else.
If the idx is invalid, print a warning.
For the case INTEL_PMC_IDX_FIXED_BTS in intel_pmu_disable_event, don't need to check the event->attr.precise_ip. Use return for the case.
--- arch/x86/events/intel/core.c | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-)
arch/x86/events/intel/core.c +28 -8
@@ -2180,17 +2180,28 @@ static void intel_pmu_disable_event(struct perf_event *event)struct hw_perf_event *hwc = &event->hw;int idx = hwc->idx; -if (idx <INTEL_PMC_IDX_FIXED ) { +switch (idx) { + case 0 ... INTEL_PMC_IDX_FIXED - 1:intel_clear_masks (event, idx);x86_pmu_disable_event (event); - }else if (idx <INTEL_PMC_IDX_FIXED_BTS ) { +break ; + case INTEL_PMC_IDX_FIXED ... INTEL_PMC_IDX_FIXED_BTS - 1:intel_clear_masks (event, idx);intel_pmu_disable_fixed (event); - }else if (idx ==INTEL_PMC_IDX_FIXED_BTS ) { +break ; +case INTEL_PMC_IDX_FIXED_BTS :intel_pmu_disable_bts ();intel_pmu_drain_bts_buffer (); - }else if (idx ==INTEL_PMC_IDX_FIXED_VLBR ) +return ; +case INTEL_PMC_IDX_FIXED_VLBR :intel_clear_masks (event, idx); +break ; +default : +intel_clear_masks (event, idx); +pr_warn ("Failed to disable the event with invalid index %d\n" , + idx); +return ; + }/* * Needs to be called after x86_pmu_disable_event,
@@ -2262,18 +2273,27 @@ static void intel_pmu_enable_event(struct perf_event *event)if (unlikely (event->attr.precise_ip))intel_pmu_pebs_enable (event); -if (idx <INTEL_PMC_IDX_FIXED ) { +switch (idx) { + case 0 ... INTEL_PMC_IDX_FIXED - 1:intel_set_masks (event, idx);__x86_pmu_enable_event (hwc,ARCH_PERFMON_EVENTSEL_ENABLE ); - }else if (idx <INTEL_PMC_IDX_FIXED_BTS ) { +break ; + case INTEL_PMC_IDX_FIXED ... INTEL_PMC_IDX_FIXED_BTS - 1:intel_set_masks (event, idx);intel_pmu_enable_fixed (event); - }else if (idx ==INTEL_PMC_IDX_FIXED_BTS ) { +break ; +case INTEL_PMC_IDX_FIXED_BTS :if (!__this_cpu_read (cpu_hw_events.enabled))return ;intel_pmu_enable_bts (hwc->config); - }else if (idx ==INTEL_PMC_IDX_FIXED_VLBR ) +break ; +case INTEL_PMC_IDX_FIXED_VLBR :intel_set_masks (event, idx); +break ; +default : +pr_warn ("Failed to enable the event with invalid index %d\n" , + idx); + } }static void intel_pmu_add_event (struct perf_event *event)
Signature · 1 line
2.17.1
PATCH v6 5/14 perf/x86/intel: Fix the name of perf METRICS
In reply to [email protected]From: Kan Liang <[email protected]>
Bit 15 of the PERF_CAPABILITIES MSR indicates that the perf METRICS feature is supported. The perf METRICS is not a PEBS feature.
Rename pebs_metrics_available perf_metrics.
The bit is not used in the current code. It will be used in a later patch.
--- arch/x86/events/perf_event.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/perf_event.h +1 -1
@@ -537,7 +537,7 @@ union perf_capabilities {*/ u64 full_width_write:1; u64 pebs_baseline:1; - u64 pebs_metrics_available:1; + u64 perf_metrics:1; u64 pebs_output_pt_available:1; };u64 capabilities;
Signature · 1 line
2.17.1
PATCH v6 2/14 perf/x86/intel: Name the global status bit in NMI handler
In reply to [email protected]From: Kan Liang <[email protected]>
Magic numbers are used in the current NMI handler for the global status bit. Use a meaningful name to replace the magic numbers to improve the readability of the code.
Remove a Tab for all GLOBAL_STATUS_* and INTEL_PMC_IDX_FIXED_BTS macros to reduce the length of the line.
--- arch/x86/events/intel/core.c | 4 ++-- arch/x86/include/asm/perf_event.h | 22 ++++++++++++---------- 2 files changed, 14 insertions(+), 12 deletions(-)
arch/x86/events/intel/core.c +2 -2
@@ -2389,7 +2389,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)/* * PEBS overflow sets bit 62 in the global status register */ -if (__test_and_clear_bit (62 , (unsigned long *)&status)) { +if (__test_and_clear_bit (GLOBAL_STATUS_BUFFER_OVF_BIT , (unsigned long *)&status)) {u64 pebs_enabled = cpuc->pebs_enabled; handled++;
@@ -2410,7 +2410,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)/* * Intel PT */ -if (__test_and_clear_bit (55 , (unsigned long *)&status)) { +if (__test_and_clear_bit (GLOBAL_STATUS_TRACE_TOPAPMI_BIT , (unsigned long *)&status)) { handled++;if (unlikely (perf_guest_cbs && perf_guest_cbs->is_in_guest () && perf_guest_cbs->handle_intel_pt_intr))
arch/x86/include/asm/perf_event.h +12 -10
@@ -225,16 +225,18 @@ struct x86_pmu_capability {* values are used by actual fixed events and higher values are used * to indicate other overflow conditions in the PERF_GLOBAL_STATUS msr. */ -#define INTEL_PMC_IDX_FIXED_BTS (INTEL_PMC_IDX_FIXED + 16) - -#define GLOBAL_STATUS_COND_CHG BIT_ULL(63) -#define GLOBAL_STATUS_BUFFER_OVF BIT_ULL(62) -#define GLOBAL_STATUS_UNC_OVF BIT_ULL(61) -#define GLOBAL_STATUS_ASIF BIT_ULL(60) -#define GLOBAL_STATUS_COUNTERS_FROZEN BIT_ULL(59) -#define GLOBAL_STATUS_LBRS_FROZEN_BIT 58 -#define GLOBAL_STATUS_LBRS_FROZEN BIT_ULL(GLOBAL_STATUS_LBRS_FROZEN_BIT) -#define GLOBAL_STATUS_TRACE_TOPAPMI BIT_ULL(55) +#define INTEL_PMC_IDX_FIXED_BTS (INTEL_PMC_IDX_FIXED + 16) + +#define GLOBAL_STATUS_COND_CHG BIT_ULL(63) +#define GLOBAL_STATUS_BUFFER_OVF_BIT 62 +#define GLOBAL_STATUS_BUFFER_OVF BIT_ULL(GLOBAL_STATUS_BUFFER_OVF_BIT) +#define GLOBAL_STATUS_UNC_OVF BIT_ULL(61) +#define GLOBAL_STATUS_ASIF BIT_ULL(60) +#define GLOBAL_STATUS_COUNTERS_FROZEN BIT_ULL(59) +#define GLOBAL_STATUS_LBRS_FROZEN_BIT 58 +#define GLOBAL_STATUS_LBRS_FROZEN BIT_ULL(GLOBAL_STATUS_LBRS_FROZEN_BIT) +#define GLOBAL_STATUS_TRACE_TOPAPMI_BIT 55 +#define GLOBAL_STATUS_TRACE_TOPAPMI BIT_ULL(GLOBAL_STATUS_TRACE_TOPAPMI_BIT)/* * We model guest LBR event tracing as another fixed-mode PMC like BTS.
Signature · 1 line
2.17.1
Peter Zijlstra
maintainerRe: [PATCH V6 03/14] perf/x86/intel: Introduce the fourth fixed counter
In reply to [email protected]Quoted 4 lines · On Fri, Jul 17, 2020 at 07:05:43AM -0700, [email protected] wrote:
/*
+ * There is no event-code assigned to the fixed-mode PMCs.
+ *
+ * For a fixed-mode PMC, which has an equivalent event on a general-purpose
Quoted 8 lines
+ * PMC, the event-code of the equivalent event is used for the fixed-mode PMC,
+ * e.g., Instr_Retired.Any and CPU_CLK_Unhalted.Core.
+ *
+ * For a fixed-mode PMC, which doesn't have an equivalent event, a
+ * pseudo-encoding is used, e.g., CPU_CLK_Unhalted.Ref and TOPDOWN.SLOTS.
+ * The pseudo event-code for a fixed-mode PMC must be0x00.
+ * The pseudo umask-code is 0x0X. The X indicates the index of the fixed
+ * counter.
Isn't it X+1 ? Such that 0x0000 is an invalid event? After all, the pseudo event for Fixed2 is 0x0300.
Quoted 3 lines
+ *
+ * The counts are available in separate MSRs:
*/
Peter Zijlstra
maintainerRe: [PATCH V6 06/14] perf/x86/intel: Use switch in intel_pmu_disable/enable_event
In reply to [email protected]Quoted 6 lines · On Fri, Jul 17, 2020 at 07:05:46AM -0700, [email protected] wrote:
From: Kan Liang <[email protected]>
Many items are checked in the intel_pmu_disable/enable_event. More items will be added later, e.g. perf metrics events.
Use switch, which is more efficient, to replace the if-else.
Perhaps explain how; does this actually generate a jump-table?
Peter Zijlstra
maintainerRe: [PATCH V6 07/14] perf/x86/intel: Generic support for hardware TopDown metrics
In reply to [email protected]arch/x86/include/asm/perf_event.h struct x86_pmu_capability { · line 257
Quoted 8 lines · On Fri, Jul 17, 2020 at 07:05:47AM -0700, [email protected] wrote:
For the event mapping, a special
0x00event code is used, which is reserved for fake events. The metric events start from umask0x10.arch/x86/include/asm/perf_event.h +5
-0+#define INTEL_PMC_IDX_METRIC_BASE (INTEL_PMC_IDX_FIXED + 16) +#define INTEL_PMC_IDX_TD_RETIRING (INTEL_PMC_IDX_METRIC_BASE + 0) +#define INTEL_PMC_IDX_TD_BAD_SPEC (INTEL_PMC_IDX_METRIC_BASE + 1) +#define INTEL_PMC_IDX_TD_FE_BOUND (INTEL_PMC_IDX_METRIC_BASE + 2) +#define INTEL_PMC_IDX_TD_BE_BOUND (INTEL_PMC_IDX_METRIC_BASE + 3)
So this is internal and we can change it around if/when needed, right?
Quoted 10 lines
arch/x86/include/asm/perf_event.h +10
-0+#define INTEL_PMC_IDX_METRIC_END INTEL_PMC_IDX_TD_BE_BOUND +#define INTEL_PMC_MSK_TOPDOWN ((0xfull << INTEL_PMC_IDX_METRIC_BASE) | \ + INTEL_PMC_MSK_FIXED_SLOTS) + +/* +* There is no event-code assigned to the TopDown events. +* +* For the slots event, use the pseudo code of the fixed counter 3. +* +* For the metric events, the pseudo event-code is 0x00.
arch/x86/include/asm/perf_event.h struct x86_pmu_capability { · line 277
Quoted 10 lines
arch/x86/include/asm/perf_event.h +10
-0+* The pseudo umask-code starts from 0x10. +*/ +#define INTEL_TD_SLOTS 0x0400/* TOPDOWN.SLOTS */ +/* Level 1 metrics */ +#define INTEL_TD_METRIC_RETIRING 0x1000/* Retiring metric */ +#define INTEL_TD_METRIC_BAD_SPEC 0x1100/* Bad speculation metric */ +#define INTEL_TD_METRIC_FE_BOUND 0x1200/* FE bound metric */ +#define INTEL_TD_METRIC_BE_BOUND 0x1300/* BE bound metric */ +#define INTEL_TD_METRIC_MAX INTEL_TD_METRIC_BE_BOUND +#define INTEL_TD_METRIC_NUM 4
But this is ABI, once we merge this, it's stuck.
Also, per how Fixed2 is 0x0300, should not Fixed16 (aka METRICS_RETIRING) be 0x1100 ?
But aside of that, are we sure the hardware will never grow a Fixed16? Or do we want to be paranoid and move the metrics events up in the pseudo event space?
Re: [PATCH V6 07/14] perf/x86/intel: Generic support for hardware TopDown metrics
Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · On Fri, Jul 17, 2020 at 07:05:47AM -0700, [email protected] wrote:
For the event mapping, a special
0x00event code is used, which is reserved for fake events. The metric events start from umask0x10.
Quoted 8 lines
arch/x86/include/asm/perf_event.h struct x86_pmu_capability { · line 257
Quoted 5 lines
arch/x86/include/asm/perf_event.h +5
-0+#define INTEL_PMC_IDX_METRIC_BASE (INTEL_PMC_IDX_FIXED + 16) +#define INTEL_PMC_IDX_TD_RETIRING (INTEL_PMC_IDX_METRIC_BASE + 0) +#define INTEL_PMC_IDX_TD_BAD_SPEC (INTEL_PMC_IDX_METRIC_BASE + 1) +#define INTEL_PMC_IDX_TD_FE_BOUND (INTEL_PMC_IDX_METRIC_BASE + 2) +#define INTEL_PMC_IDX_TD_BE_BOUND (INTEL_PMC_IDX_METRIC_BASE + 3)So this is internal and we can change it around if/when needed, right?
Yes
Quoted 23 lines
Quoted 10 lines
arch/x86/include/asm/perf_event.h +10
-0+#define INTEL_PMC_IDX_METRIC_END INTEL_PMC_IDX_TD_BE_BOUND +#define INTEL_PMC_MSK_TOPDOWN ((0xfull << INTEL_PMC_IDX_METRIC_BASE) | \ + INTEL_PMC_MSK_FIXED_SLOTS) + +/* +* There is no event-code assigned to the TopDown events. +* +* For the slots event, use the pseudo code of the fixed counter 3. +* +* For the metric events, the pseudo event-code is 0x00. arch/x86/include/asm/perf_event.h struct x86_pmu_capability { · line 277
Quoted 10 lines
arch/x86/include/asm/perf_event.h +10
-0+* The pseudo umask-code starts from 0x10. +*/ +#define INTEL_TD_SLOTS 0x0400/* TOPDOWN.SLOTS */ +/* Level 1 metrics */ +#define INTEL_TD_METRIC_RETIRING 0x1000/* Retiring metric */ +#define INTEL_TD_METRIC_BAD_SPEC 0x1100/* Bad speculation metric */ +#define INTEL_TD_METRIC_FE_BOUND 0x1200/* FE bound metric */ +#define INTEL_TD_METRIC_BE_BOUND 0x1300/* BE bound metric */ +#define INTEL_TD_METRIC_MAX INTEL_TD_METRIC_BE_BOUND +#define INTEL_TD_METRIC_NUM 4But this is ABI, once we merge this, it's stuck.
Quoted 7 lines
Also, per how Fixed2 is
0x0300, should not Fixed16 (akaMETRICS_RETIRING) be0x1100?But aside of that, are we sure the hardware will never grow a Fixed16? Or do we want to be paranoid and move the metrics events up in the pseudo event space?
Right, we cannot guarantee that the number of fixed counters is always less than 16.
I will move the metrics events to the middle of the pseudo event space.
+#define INTEL_TD_METRIC_RETIRING 0x8000 /* Retiring metric */
+#define INTEL_TD_METRIC_BAD_SPEC 0x8100 /* Bad speculation metric */
+#define INTEL_TD_METRIC_FE_BOUND 0x8200 /* FE bound metric */
+#define INTEL_TD_METRIC_BE_BOUND 0x8300 /* BE bound metric */
Thanks,
Kan
Re: [PATCH V6 03/14] perf/x86/intel: Introduce the fourth fixed counter
In reply to Peter ZijlstraQuoted 14 lines · Peter Zijlstra
Quoted 4 lines · On Fri, Jul 17, 2020 at 07:05:43AM -0700, [email protected] wrote:
/*
+ * There is no event-code assigned to the fixed-mode PMCs.
+ *
+ * For a fixed-mode PMC, which has an equivalent event on a general-purposeQuoted 8 lines
+ * PMC, the event-code of the equivalent event is used for the fixed-mode PMC,
+ * e.g., Instr_Retired.Any and CPU_CLK_Unhalted.Core.
+ *
+ * For a fixed-mode PMC, which doesn't have an equivalent event, a
+ * pseudo-encoding is used, e.g., CPU_CLK_Unhalted.Ref and TOPDOWN.SLOTS.
+ * The pseudo event-code for a fixed-mode PMC must be0x00.
+ * The pseudo umask-code is 0x0X. The X indicates the index of the fixed
+ * counter.
Quoted 2 lines
Isn't it X+1 ? Such that
0x0000is an invalid event? After all, the pseudo event for Fixed2 is0x0300.
I will fix the inaccurate description as below in V7.
+ * The pseudo umask-code is 0xX. The X equals the index of the fixed
+ * counter + 1, e.g., the fixed counter 2 has the pseudo-encoding 0x0300.
Thanks,
Kan
Re: [PATCH V6 06/14] perf/x86/intel: Use switch in intel_pmu_disable/enable_event
In reply to Peter ZijlstraQuoted 9 lines · Peter Zijlstra
Quoted 6 lines · On Fri, Jul 17, 2020 at 07:05:46AM -0700, [email protected] wrote:
From: Kan Liang <[email protected]>
Many items are checked in the intel_pmu_disable/enable_event. More items will be added later, e.g. perf metrics events.
Use switch, which is more efficient, to replace the if-else.
Perhaps explain how; does this actually generate a jump-table?
As my understanding, some optimizing compilers may compile a switch statement into a jump-table. So it depends.
Another advantage of switch statement is to improve the readability of the code. I think I missed it in the commit message.
I will change the commit message as below in V7.
"Currently, the if-else is used in the intel_pmu_disable/enable_event to check the type of an event. It works well, but with more and more types added later, e.g., perf metrics, compared to the switch statement, the if-else may impair the readability of the code.
Also, some optimizing compilers may compile a switch statement into a jump-table which is more efficient than if-else.
Use switch to replace the if-else in the intel_pmu_disable/enable_event()."
Thanks,
Kan
Peter Zijlstra
maintainerRe: [PATCH V6 07/14] perf/x86/intel: Generic support for hardware TopDown metrics
In reply to [email protected]Quoted 48 lines · On Fri, Jul 17, 2020 at 07:05:47AM -0700, [email protected] wrote:
(quoted diff) +33
-0@@ -1031,6 +1034,35 @@ int x86_schedule_events(struct cpu_hw_events *cpuc, int n, int *assign) return unsched ? -EINVAL : 0; } +static int add_nr_metric_event(struct cpu_hw_events *cpuc, + struct perf_event *event, + int *max_count, bool sibling) +{ + /* The TopDown metrics events cannot be shared. */ + if (is_metric_event(event) && + (++cpuc->n_metric_event > INTEL_TD_METRIC_NUM)) { + cpuc->n_metric_event--; + return -EINVAL; + } + + /* + * Take the accepted metrics events into account for leader event. + */ + if (!sibling) + *max_count += cpuc->n_metric_event; + else if (is_metric_event(event)) + (*max_count)++; + + return 0; +} + +static void del_nr_metric_event(struct cpu_hw_events *cpuc, + struct perf_event *event) +{ + if (is_metric_event(event)) + cpuc->n_metric_event--; +} + /* * dogrp: true if must collect siblings events (group) * returns total number of events and error code@@ -1066,6 +1098,10 @@ static int collect_events(struct cpu_hw_events *cpuc, struct perf_event *leader, cpuc->pebs_output = is_pebs_pt(leader) + 1; } + if (x86_pmu.intel_cap.perf_metrics && + add_nr_metric_event(cpuc, leader, &max_count, false)) + return -EINVAL; + if (is_x86_event(leader)) { if (n >= max_count) return -EINVAL;
Review static int collect_events(struct cpu_hw_events *cpuc, struct perf_event *leader, · line 1127
Quoted 10 lines
(quoted diff) +4
-0@@ -1082,6 +1118,10 @@ static int collect_events(struct cpu_hw_events *cpuc, struct perf_event *leader, event->state <= PERF_EVENT_STATE_OFF) continue; + if (x86_pmu.intel_cap.perf_metrics && + add_nr_metric_event(cpuc, event, &max_count, true)) + return -EINVAL; + if (n >= max_count) return -EINVAL;
Something like so perhaps ?
arch/x86/events/core.c +27 -37
@@ -1035,24 +1035,14 @@ int x86_schedule_events(struct cpu_hw_ev }static int add_nr_metric_event (struct cpu_hw_events *cpuc, -struct perf_event *event, -int *max_count,bool sibling) +struct perf_event *event) { -/* The TopDown metrics events cannot be shared. */ -if (is_metric_event (event) && - (++cpuc->n_metric_event >INTEL_TD_METRIC_NUM )) { - cpuc->n_metric_event--; -return -EINVAL ; +if (is_metric_event (event)) { +if (cpuc->n_metric ==INTEL_TD_METRIC_NUM ) +return -EINVAL ; + cpuc->n_metric++; } -/* -* Take the accepted metrics events into account for leader event. -*/ -if (!sibling) - *max_count += cpuc->n_metric_event; -else if (is_metric_event (event)) - (*max_count)++; -return 0 ; }
@@ -1060,7 +1050,24 @@ static void del_nr_metric_event(struct c struct perf_event *event) {if (is_metric_event (event)) - cpuc->n_metric_event--; + cpuc->n_metric--; +} + +static int collect_event (struct cpu_hw_events *cpuc,struct perf_event *event, +int max_count,int n) +{ + +if (x86_pmu.intel_cap.perf_metrics &&add_nr_metric_event (cpuc, event)) +return -EINVAL ; + +if (n >= max_count + cpuc->n_metric) +return -EINVAL ; + + cpuc->event_list[n] = event; +if (is_counter_pair (&event->hw)) + cpuc->n_pair++; + +return 0 ; }/*
@@ -1098,37 +1105,20 @@ static int collect_events(struct cpu_hw_ cpuc->pebs_output =is_pebs_pt (leader) +1 ; } -if (x86_pmu.intel_cap.perf_metrics && -add_nr_metric_event (cpuc, leader, &max_count, false)) +if (is_x86_event (leader) &&collect_event (cpuc, leader, max_count, n))return -EINVAL ; + n++; -if (is_x86_event (leader)) { -if (n >= max_count) -return -EINVAL ; - cpuc->event_list[n] = leader; - n++; -if (is_counter_pair (&leader->hw)) - cpuc->n_pair++; - }if (!dogrp)return n;for_each_sibling_event (event, leader) { -if (!is_x86_event (event) || - event->state <=PERF_EVENT_STATE_OFF ) +if (!is_x86_event (event) || event->state <=PERF_EVENT_STATE_OFF )continue ; -if (x86_pmu.intel_cap.perf_metrics && -add_nr_metric_event (cpuc, event, &max_count, true)) -return -EINVAL ; - -if (n >= max_count) +if (collect_event (cpuc, event, max_count, n))return -EINVAL ; - - cpuc->event_list[n] = event; n++; -if (is_counter_pair (&event->hw)) - cpuc->n_pair++; }return n; }
arch/x86/events/perf_event.h +1 -1
@@ -313,7 +313,7 @@ struct cpu_hw_events {* Perf Metrics */ /* number of accepted metrics events */ -int n_metric_event; +int n_metric;/* * AMD specific bits
Peter Zijlstra
maintainerRe: [PATCH V6 09/14] perf/x86/intel: Support TopDown metrics on Ice Lake
In reply to [email protected]Quoted 56 lines · On Fri, Jul 17, 2020 at 07:05:49AM -0700, [email protected] wrote:
arch/x86/events/intel/core.c +56
-0+static inline u64 icl_get_metrics_event_value (u64 metric,u64 slots,int idx) +{ +u32 val; + +/* +* The metric is reported as an 8bit integer fraction +* suming up to 0xff. +* slots-in-metric = (Metric / 0xff) * slots +*/ + val = (metric >> ((idx -INTEL_PMC_IDX_METRIC_BASE ) *8 )) &0xff ; +return mul_u64_u32_div (slots, val,0xff ); +} + +static void __icl_update_topdown_event (struct perf_event *event, +u64 slots,u64 metrics) +{ +int idx = event->hw.idx; +u64 delta; + +if (is_metric_idx (idx)) + delta =icl_get_metrics_event_value (metrics, slots, idx); +else + delta = slots; + +local64_add (delta, &event->count); +} + +/* +* Update all active Topdown events. +* +* The PERF_METRICS and Fixed counter 3 are read separately. The values may be +* modify by a NMI. PMU has to be disabled before calling this function. +*/ +static u64 icl_update_topdown_event (struct perf_event *event) +{ +struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +struct perf_event *other; +u64 slots, metrics; +int idx; + +/* read Fixed counter 3 */ +rdpmcl ((3 |INTEL_PMC_FIXED_RDPMC_BASE ), slots); +if (!slots) +return 0 ; + +/* read PERF_METRICS */ +rdpmcl (INTEL_PMC_FIXED_RDPMC_METRICS , metrics); + +for_each_set_bit (idx, cpuc->active_mask,INTEL_PMC_IDX_TD_BE_BOUND +1 ) { +if (!is_topdown_idx (idx)) +continue ; + other = cpuc->events[idx]; +__icl_update_topdown_event (other, slots, metrics); + } + +/*
arch/x86/events/intel/core.c static void intel_pmu_del_event(struct perf_event *event) · line 2334
Quoted 10 lines
arch/x86/events/intel/core.c +10
-0+* Check and update this event, which may have been cleared +* in active_mask e.g. x86_pmu_stop() +*/ +if (event && !test_bit (event->hw.idx, cpuc->active_mask)) +__icl_update_topdown_event (event, slots, metrics); + +/* +* Software is recommended to periodically clear both registers +* in order to maintain accurate measurements, which is required for +* certain scenarios that involve sampling metrics at high rates.
I'll maintain that that statement is clear as mud and therefore useless.
arch/x86/events/intel/core.c static void intel_pmu_del_event(struct perf_event *event) · line 2342
Quoted 8 lines
arch/x86/events/intel/core.c +8
-0+* Software should always write fixed counter 3 before write to +* PERF_METRICS. +*/ +wrmsrl (MSR_CORE_PERF_FIXED_CTR3 ,0 ); +wrmsrl (MSR_PERF_METRICS ,0 ); + +return slots; +}
So in the normal case, this ends up calling into this function _5_ times, once for each event. Now the first time, it will actually do something useful, but the other 4 times it's just wasting cycles.
Also, that for_each_set_bit() loop, trying to find the events to update...
Can't we, instead, make the SLOTS update advance 5 running counters in cpuc and feed the events off of that?
Peter Zijlstra
maintainerRe: [PATCH V6 11/14] perf/x86/intel: Disable sample-read the slots and metrics events
In reply to [email protected]Quoted 8 lines · On Fri, Jul 17, 2020 at 07:05:51AM -0700, [email protected] wrote:
From: Kan Liang <[email protected]>
Users fail to sample-read the slots and metrics events, e.g., perf record -e '{slots, topdown-retiring}:S'.
When reading the metrics event, the fixed counter 3 (slots) has to be reset, which impacts the sampling of the slots event.
Quoted 8 lines
Add a specific
validate_group()support to reject the case and error out for Ice Lake.An alternative fix may unconditionally disable slots sampling, but it's not a decent fix. Users may want to only sample the slot events without the topdown metrics events.
Signed-off-by Kan Liang <[email protected]>
I'm confused by this; it doesn't make sense.
Should not patch 7 have something like the below instead?
Also, I think there is a bug when we create a group like this and then kill the leader, in that case the core code will 'promote' the sibling metric events to their own individual events, see perf_group_detach().
We need additional code to move those events into unrecoverable ERROR state. A new group_caps flag could indicate this promotion isn't allowed.
arch/x86/events/intel/core.c +16 -2
@@ -3441,8 +3441,22 @@ static int intel_pmu_hw_config(struct pe* A flag PERF_X86_EVENT_TOPDOWN is applied for the case. */ if (x86_pmu.intel_cap.perf_metrics &&is_topdown_event (event)) { -if (is_metric_event (event) &&is_sampling_event (event)) -return -EINVAL ; + +if (is_metric_event (event)) { +struct perf_event *leader = event->group_leader; + +if (is_sampling_event (event)) +return -EINVAL ; + +if (leader == event) +return -EINVAL ; + +if (!is_slots_event (leader)) +return -EINVAL ; + +if (is_sampling_event (leader)) +return -EINVAL ; + }if (!is_sampling_event (event)) {if (event->attr.config1 !=0 )
Re: [PATCH V6 07/14] perf/x86/intel: Generic support for hardware TopDown metrics
In reply to Peter ZijlstraQuoted 121 lines · Peter Zijlstra
Quoted 48 lines · On Fri, Jul 17, 2020 at 07:05:47AM -0700, [email protected] wrote:
(quoted diff) +33
-0@@ -1031,6 +1034,35 @@ int x86_schedule_events(struct cpu_hw_events *cpuc, int n, int *assign) return unsched ? -EINVAL : 0; } +static int add_nr_metric_event(struct cpu_hw_events *cpuc, + struct perf_event *event, + int *max_count, bool sibling) +{ + /* The TopDown metrics events cannot be shared. */ + if (is_metric_event(event) && + (++cpuc->n_metric_event > INTEL_TD_METRIC_NUM)) { + cpuc->n_metric_event--; + return -EINVAL; + } + + /* + * Take the accepted metrics events into account for leader event. + */ + if (!sibling) + *max_count += cpuc->n_metric_event; + else if (is_metric_event(event)) + (*max_count)++; + + return 0; +} + +static void del_nr_metric_event(struct cpu_hw_events *cpuc, + struct perf_event *event) +{ + if (is_metric_event(event)) + cpuc->n_metric_event--; +} + /* * dogrp: true if must collect siblings events (group) * returns total number of events and error code@@ -1066,6 +1098,10 @@ static int collect_events(struct cpu_hw_events *cpuc, struct perf_event *leader, cpuc->pebs_output = is_pebs_pt(leader) + 1; } + if (x86_pmu.intel_cap.perf_metrics && + add_nr_metric_event(cpuc, leader, &max_count, false)) + return -EINVAL; + if (is_x86_event(leader)) { if (n >= max_count) return -EINVAL;Review static int collect_events(struct cpu_hw_events *cpuc, struct perf_event *leader, · line 1127
Quoted 10 lines
(quoted diff) +4
-0@@ -1082,6 +1118,10 @@ static int collect_events(struct cpu_hw_events *cpuc, struct perf_event *leader, event->state <= PERF_EVENT_STATE_OFF) continue; + if (x86_pmu.intel_cap.perf_metrics && + add_nr_metric_event(cpuc, event, &max_count, true)) + return -EINVAL; + if (n >= max_count) return -EINVAL;Something like so perhaps ?
arch/x86/events/core.c +23
-16@@ -1035,24 +1035,14 @@ int x86_schedule_events(struct cpu_hw_ev }static int add_nr_metric_event (struct cpu_hw_events *cpuc, -struct perf_event *event, -int *max_count,bool sibling) +struct perf_event *event) { -/* The TopDown metrics events cannot be shared. */ -if (is_metric_event (event) && - (++cpuc->n_metric_event >INTEL_TD_METRIC_NUM )) { - cpuc->n_metric_event--; -return -EINVAL ; +if (is_metric_event (event)) { +if (cpuc->n_metric ==INTEL_TD_METRIC_NUM ) +return -EINVAL ; + cpuc->n_metric++; } -/* -* Take the accepted metrics events into account for leader event. -*/ -if (!sibling) - *max_count += cpuc->n_metric_event; -else if (is_metric_event (event)) - (*max_count)++; -return 0 ; }@@ -1060,7 +1050,24 @@ static void del_nr_metric_event(struct c struct perf_event *event) {if (is_metric_event (event)) - cpuc->n_metric_event--; + cpuc->n_metric--; +} + +static int collect_event (struct cpu_hw_events *cpuc,struct perf_event *event, +int max_count,int n) +{ + +if (x86_pmu.intel_cap.perf_metrics &&add_nr_metric_event (cpuc, event)) +return -EINVAL ; + +if (n >= max_count + cpuc->n_metric) +return -EINVAL ; + + cpuc->event_list[n] = event; +if (is_counter_pair (&event->hw)) + cpuc->n_pair++; + +return 0 ; }/*
arch/x86/events/core.c static int collect_events(struct cpu_hw_ · line 1110
Quoted 8 lines
arch/x86/events/core.c +2
-2@@ -1098,37 +1105,20 @@ static int collect_events(struct cpu_hw_ cpuc->pebs_output =is_pebs_pt (leader) +1 ; } -if (x86_pmu.intel_cap.perf_metrics && -add_nr_metric_event (cpuc, leader, &max_count, false)) +if (is_x86_event (leader) &&collect_event (cpuc, leader, max_count, n))return -EINVAL ; + n++;
If a leader is not an x86 event, n will be mistakenly increased. But is it possible that a leader is not an x86 event here?
Seems impossible for now. A SW event cannot be a leader for a mix group. We don't allow the core PMU and the perf_invalid_context PMU in the same group. If so, I think it deserves a comment, in case the situation changes later, e.g.,
+ if (is_x86_event(leader) && collect_event(cpuc, leader, max_count, n))
return -EINVAL;
+ /*
+ * Currently, for a x86 core event group, the leader must be a
+ * x86 core event. A SW event cannot be a leader for a mix
+ * group. We don't allow the core PMU and the perf_invalid_contex +
* PMU in the same group.
+ */
+ n++;
Thanks,
Kan
Quoted 45 lines
arch/x86/events/core.c +2
-19-if (is_x86_event (leader)) { -if (n >= max_count) -return -EINVAL ; - cpuc->event_list[n] = leader; - n++; -if (is_counter_pair (&leader->hw)) - cpuc->n_pair++; - }if (!dogrp)return n;for_each_sibling_event (event, leader) { -if (!is_x86_event (event) || - event->state <=PERF_EVENT_STATE_OFF ) +if (!is_x86_event (event) || event->state <=PERF_EVENT_STATE_OFF )continue ; -if (x86_pmu.intel_cap.perf_metrics && -add_nr_metric_event (cpuc, event, &max_count, true)) -return -EINVAL ; - -if (n >= max_count) +if (collect_event (cpuc, event, max_count, n))return -EINVAL ; - - cpuc->event_list[n] = event; n++; -if (is_counter_pair (&event->hw)) - cpuc->n_pair++; }return n; }arch/x86/events/perf_event.h +1
-1@@ -313,7 +313,7 @@ struct cpu_hw_events {* Perf Metrics */ /* number of accepted metrics events */ -int n_metric_event; +int n_metric;/* * AMD specific bits
Re: [PATCH V6 09/14] perf/x86/intel: Support TopDown metrics on Ice Lake
In reply to Peter ZijlstraQuoted 84 lines · Peter Zijlstra
Quoted 56 lines · On Fri, Jul 17, 2020 at 07:05:49AM -0700, [email protected] wrote:
arch/x86/events/intel/core.c +56
-0+static inline u64 icl_get_metrics_event_value (u64 metric,u64 slots,int idx) +{ +u32 val; + +/* +* The metric is reported as an 8bit integer fraction +* suming up to 0xff. +* slots-in-metric = (Metric / 0xff) * slots +*/ + val = (metric >> ((idx -INTEL_PMC_IDX_METRIC_BASE ) *8 )) &0xff ; +return mul_u64_u32_div (slots, val,0xff ); +} + +static void __icl_update_topdown_event (struct perf_event *event, +u64 slots,u64 metrics) +{ +int idx = event->hw.idx; +u64 delta; + +if (is_metric_idx (idx)) + delta =icl_get_metrics_event_value (metrics, slots, idx); +else + delta = slots; + +local64_add (delta, &event->count); +} + +/* +* Update all active Topdown events. +* +* The PERF_METRICS and Fixed counter 3 are read separately. The values may be +* modify by a NMI. PMU has to be disabled before calling this function. +*/ +static u64 icl_update_topdown_event (struct perf_event *event) +{ +struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +struct perf_event *other; +u64 slots, metrics; +int idx; + +/* read Fixed counter 3 */ +rdpmcl ((3 |INTEL_PMC_FIXED_RDPMC_BASE ), slots); +if (!slots) +return 0 ; + +/* read PERF_METRICS */ +rdpmcl (INTEL_PMC_FIXED_RDPMC_METRICS , metrics); + +for_each_set_bit (idx, cpuc->active_mask,INTEL_PMC_IDX_TD_BE_BOUND +1 ) { +if (!is_topdown_idx (idx)) +continue ; + other = cpuc->events[idx]; +__icl_update_topdown_event (other, slots, metrics); + } + +/* arch/x86/events/intel/core.c static void intel_pmu_del_event(struct perf_event *event) · line 2334
Quoted 10 lines
arch/x86/events/intel/core.c +10
-0+* Check and update this event, which may have been cleared +* in active_mask e.g. x86_pmu_stop() +*/ +if (event && !test_bit (event->hw.idx, cpuc->active_mask)) +__icl_update_topdown_event (event, slots, metrics); + +/* +* Software is recommended to periodically clear both registers +* in order to maintain accurate measurements, which is required for +* certain scenarios that involve sampling metrics at high rates. I'll maintain that that statement is clear as mud and therefore useless.
arch/x86/events/intel/core.c static void intel_pmu_del_event(struct perf_event *event) · line 2342
Quoted 8 lines
arch/x86/events/intel/core.c +8
-0+* Software should always write fixed counter 3 before write to +* PERF_METRICS. +*/ +wrmsrl (MSR_CORE_PERF_FIXED_CTR3 ,0 ); +wrmsrl (MSR_PERF_METRICS ,0 ); + +return slots; +}So in the normal case, this ends up calling into this function _5_ times, once for each event. Now the first time, it will actually do something useful, but the other 4 times it's just wasting cycles.
Quoted 6 lines
Also, that
for_each_set_bit()loop, trying to find the events to update...Can't we, instead, make the SLOTS update advance 5 running counters in cpuc and feed the events off of that?
Patch 13 forces the slots event to be part of a metric group. In patch 7, for a metric group, we only update the values once with slots event. I think the normal case mentioned above should not happen.
+ /* Only need to call update_topdown_event() once for group read. */
+ if ((cpuc->txn_flags & PERF_PMU_TXN_READ) &&
+ !is_slots_event(event))
+ return;
+
+ perf_pmu_disable(event->pmu);
+ x86_pmu.update_topdown_event(event);
+ perf_pmu_enable(event->pmu);
Thanks,
Kan
[email protected]
maintainerRe: [PATCH V6 07/14] perf/x86/intel: Generic support for hardware TopDown metrics
In reply to Liang, KanQuoted 12 lines · Kan
arch/x86/events/core.c static int collect_events(struct cpu_hw_ · line 1109
Quoted 8 lines · Peter Zijlstra
arch/x86/events/core.c +2
-2@@ -1098,37 +1105,20 @@ static int collect_events(struct cpu_hw_ cpuc->pebs_output =is_pebs_pt (leader) +1 ; } -if (x86_pmu.intel_cap.perf_metrics && -add_nr_metric_event (cpuc, leader, &max_count, false)) +if (is_x86_event (leader) &&collect_event (cpuc, leader, max_count, n))return -EINVAL ; + n++;If a leader is not an x86 event, n will be mistakenly increased. But is it possible that a leader is not an x86 event here?
You're right, and yes that can happen, see the move_group=1 case in sys_perf_event_open().
if (is_x86_event(leader)) {
if (collect_event(cpuc, leader, max_count, n))
return -EINVAL;
n++;
}it is then..
[email protected]
maintainerRe: [PATCH V6 09/14] perf/x86/intel: Support TopDown metrics on Ice Lake
In reply to Liang, KanReview
Quoted 12 lines · Kan
Patch 13 forces the slots event to be part of a metric group. In patch 7, for a metric group, we only update the values once with slots event. I think the normal case mentioned above should not happen.
(quoted diff) +8
-0+ /* Only need to call update_topdown_event() once for group read. */ + if ((cpuc->txn_flags & PERF_PMU_TXN_READ) && + !is_slots_event(event)) + return; + + perf_pmu_disable(event->pmu); + x86_pmu.update_topdown_event(event); + perf_pmu_enable(event->pmu);
Ah, I missed that.
That also requires SLOTS to be the leader so that it will be the first read. Did we enforce that somewhere?
Re: [PATCH V6 09/14] perf/x86/intel: Support TopDown metrics on Ice Lake
Quoted 17 lines · On 7/21/2020 10:31 AM, [email protected] wrote:
Review
Quoted 12 lines · Kan
Patch 13 forces the slots event to be part of a metric group. In patch 7, for a metric group, we only update the values once with slots event. I think the normal case mentioned above should not happen.
(quoted diff) +8
-0+ /* Only need to call update_topdown_event() once for group read. */ + if ((cpuc->txn_flags & PERF_PMU_TXN_READ) && + !is_slots_event(event)) + return; + + perf_pmu_disable(event->pmu); + x86_pmu.update_topdown_event(event); + perf_pmu_enable(event->pmu);Ah, I missed that.
Quoted 2 lines
That also requires SLOTS to be the leader so that it will be the first read. Did we enforce that somewhere?
I think it's missed in the current patch set. The suggested code in patch 11 forces SLOTS to be the leader. I will apply it in V7.
Thanks,
Kan
Re: [PATCH V6 11/14] perf/x86/intel: Disable sample-read the slots and metrics events
In reply to Peter ZijlstraQuoted 16 lines · Peter Zijlstra
Quoted 15 lines · On Fri, Jul 17, 2020 at 07:05:51AM -0700, [email protected] wrote:
From: Kan Liang <[email protected]>
Users fail to sample-read the slots and metrics events, e.g., perf record -e '{slots, topdown-retiring}:S'.
When reading the metrics event, the fixed counter 3 (slots) has to be reset, which impacts the sampling of the slots event.
Add a specific
validate_group()support to reject the case and error out for Ice Lake.An alternative fix may unconditionally disable slots sampling, but it's not a decent fix. Users may want to only sample the slot events without the topdown metrics events.
Quoted 8 lines
Quoted 1 line
Signed-off-by Kan Liang <[email protected]>I'm confused by this; it doesn't make sense.
Should not patch 7 have something like the below instead?
Quoted 1 line
Also, I think there is a bug when we create a group like this and then
kill the leader, in that case the core code will 'promote' the sibling metric events to their own individual events, see
perf_group_detach().
I'm trying to produce the bug mentioned above, but I'm not sure under what situation, the core code will 'promote' the sibling metric events?
I tried the suggested code below. It works well for the sample-read case. Perf tool errors out as expected.
perf record -e '{slots,topdown-fe-bound}:S' sleep 1
Error:
The sys_perf_event_open() syscall returned with 22 (Invalid argument)
for event (topdown-fe-bound).
/bin/dmesg | grep -i perf may provide additional information.
Thanks,
Kan
Quoted 33 lines
We need additional code to move those events into unrecoverable ERROR state. A new
group_capsflag could indicate this promotion isn't allowed.arch/x86/events/intel/core.c +16
-2@@ -3441,8 +3441,22 @@ static int intel_pmu_hw_config(struct pe* A flag PERF_X86_EVENT_TOPDOWN is applied for the case. */ if (x86_pmu.intel_cap.perf_metrics &&is_topdown_event (event)) { -if (is_metric_event (event) &&is_sampling_event (event)) -return -EINVAL ; + +if (is_metric_event (event)) { +struct perf_event *leader = event->group_leader; + +if (is_sampling_event (event)) +return -EINVAL ; + +if (leader == event) +return -EINVAL ; + +if (!is_slots_event (leader)) +return -EINVAL ; + +if (is_sampling_event (leader)) +return -EINVAL ; + }if (!is_sampling_event (event)) {if (event->attr.config1 !=0 )
Re: [PATCH V6 09/14] perf/x86/intel: Support TopDown metrics on Ice Lake
In reply to Peter ZijlstraQuoted 5 lines
Also, that
for_each_set_bit()loop, trying to find the events to update...Can't we, instead, make the SLOTS update advance 5 running counters in cpuc and feed the events off of that?
The original patches implemented this through a perf transaction and a cache. I think what you're suggesting is similar to the old cache.
-Andi
Peter Zijlstra
maintainerRe: [PATCH V6 11/14] perf/x86/intel: Disable sample-read the slots and metrics events
In reply to Liang, KanQuoted 2 lines · Kan
I'm trying to produce the bug mentioned above, but I'm not sure under what situation, the core code will 'promote' the sibling metric events?
Create an event group:
fd = sys_perf_event_open(); fd1 = sys_perf_event_open(.group_fd = fd); ...
then kill the group leader:
close(fd);
Then the sibling events: fdN, should get promoted. Ideally try and use fd1 in a read or so after this.
Quoted 2 lines
I tried the suggested code below. It works well for the sample-read case. Perf tool errors out as expected.
I'm not sure you can create that case with perf-tool, it's a bit of a daft thing to do, so you'll have to write a custom program.
Peter Zijlstra
maintainerRe: [PATCH V6 09/14] perf/x86/intel: Support TopDown metrics on Ice Lake
In reply to Andi KleenQuoted 8 lines · Andi Kleen
Quoted 5 lines
Also, that
for_each_set_bit()loop, trying to find the events to update...Can't we, instead, make the SLOTS update advance 5 running counters in cpuc and feed the events off of that?
The original patches implemented this through a perf transaction and a cache. I think what you're suggesting is similar to the old cache.
I can't remember :-) Anyway, the TXN_READ thing Kan pointed out should work fine, I just missed it when reading through the code.
Re: [PATCH V6 11/14] perf/x86/intel: Disable sample-read the slots and metrics events
In reply to Peter ZijlstraQuoted 22 lines · Peter Zijlstra
Quoted 8 lines · On Fri, Jul 17, 2020 at 07:05:51AM -0700, [email protected] wrote:
From: Kan Liang <[email protected]>
Users fail to sample-read the slots and metrics events, e.g., perf record -e '{slots, topdown-retiring}:S'.
When reading the metrics event, the fixed counter 3 (slots) has to be reset, which impacts the sampling of the slots event.
Quoted 8 lines
Add a specific
validate_group()support to reject the case and error out for Ice Lake.An alternative fix may unconditionally disable slots sampling, but it's not a decent fix. Users may want to only sample the slot events without the topdown metrics events.
Signed-off-by Kan Liang <[email protected]>I'm confused by this; it doesn't make sense.
Should not patch 7 have something like the below instead?
Quoted 7 lines
Also, I think there is a bug when we create a group like this and then kill the leader, in that case the core code will 'promote' the sibling metric events to their own individual events, see
perf_group_detach().We need additional code to move those events into unrecoverable ERROR state. A new
group_capsflag could indicate this promotion isn't allowed.
Yes, I can produce the bug with a custom program. The below patch can fix it. I will add it into the upcoming V7 patch set.
From c90d4c19b24392e6bd9ff3c6d23d0d300bfa774f Mon Sep 17 00:00:00 2001
From: Kan Liang <[email protected]>
Date: Wed, 22 Jul 2020 12:06:32 -0700
Subject: [PATCH] perf/core: Add a new PERF_EV_CAP_COEXIST event capability
Current perf assumes that group events are independent. Close an event doesn't impact the value of the other events in the same group. If the closed event is a member, after the event closure, other events are running like a group. If the closed event is a leader, after the closure, other events are running as singleton events.
However, the assumption is not correct anymore, e.g., the TopDown slots and metrics events. The slots and metrics events must coexist in the same group. If the slots event is closed, the value for the metrics events is invalid.
Add a new PERF_EV_CAP_COEXIST event capability to indicate the relationship among group events.
If any event with the flag is detached from the group, split the group into singleton events, and move the events, which also have the flag, to the unrecoverable ERROR state.
The leader of a PERF_EV_CAP_COEXIST group has to be updated at last. Move perf_event__header_size(leader); to the end of perf_group_detach().
--- arch/x86/events/intel/core.c | 2 ++ include/linux/perf_event.h | 5 ++++ kernel/events/core.c | 52 +++++++++++++++++++++++++++++++++--- 3 files changed, 55 insertions(+), 4 deletions(-)
arch/x86/events/intel/core.c +2 -0
@@ -3659,6 +3659,8 @@ static int intel_pmu_hw_config(struct perf_event *event)return -EINVAL ; event->hw.flags |=PERF_X86_EVENT_TOPDOWN ; + + event->event_caps |=PERF_EV_CAP_COEXIST ; } }
include/linux/perf_event.h +0 -0
@@ -585,9 +585,14 @@ typedef void (*perf_overflow_handler_t)(struct perf_event *,* PERF_EV_CAP_SOFTWARE: Is a software event. * PERF_EV_CAP_READ_ACTIVE_PKG: A CPU event (or cgroup event) that can
be read
* from any CPU in the package where it is active.
+ * PERF_EV_CAP_COEXIST: An event with this flag must coexist with other sibling
+ * events, which have the same flag. If any event with the flag is detached
+ * from the group, split the group into singleton events, and move the events
+ * with the flag to the unrecoverable ERROR state.
*/
#define PERF_EV_CAP_SOFTWARE BIT(0)
#define PERF_EV_CAP_READ_ACTIVE_PKG BIT(1)
+#define PERF_EV_CAP_COEXIST BIT(2)
#define SWEVENT_HLIST_BITS 8
#define SWEVENT_HLIST_SIZE (1 << SWEVENT_HLIST_BITS)
kernel/events/core.c +48 -4
91 lines not on this page · raw message
Signature · 1 line
2.17.1
Quoted 15 lines
arch/x86/events/intel/core.c +6
-2@@ -3441,8 +3441,22 @@ static int intel_pmu_hw_config(struct pe* A flag PERF_X86_EVENT_TOPDOWN is applied for the case. */ if (x86_pmu.intel_cap.perf_metrics &&is_topdown_event (event)) { -if (is_metric_event (event) &&is_sampling_event (event)) -return -EINVAL ; + +if (is_metric_event (event)) { +struct perf_event *leader = event->group_leader; + +if (is_sampling_event (event)) +return -EINVAL ;
arch/x86/events/intel/core.c static int intel_pmu_hw_config(struct pe · line 3459
Quoted 10 lines
arch/x86/events/intel/core.c +10
-0+ +if (leader == event) +return -EINVAL ; + +if (!is_slots_event (leader)) +return -EINVAL ; + +if (is_sampling_event (leader)) +return -EINVAL ; + }
The leader doesn't need to be a slots event, e.g., {cycles, slots, topdown_fe_bound} is also a valid group. But the slots event must be before all metric events.
So I will also use the below codes to replace the above one in V7.
+ if (is_metric_event(event)) {
+ struct perf_event *leader = event->group_leader;
+ struct perf_event *sibling;
+
+ /* The metric events don't support sampling. */
+ if (is_sampling_event(event))
+ return -EINVAL;
+
+ /* The metric events cannot be a group leader. */
+ if (leader == event)
+ return -EINVAL;
+
+ /*
+ * The slots event cannot be the leader of a topdown
+ * sample-read group, e.g., {slots, topdown-retiring}:S
+ */
+ if (is_slots_event(leader) && is_sampling_event(leader))
+ return -EINVAL;
+
+ /*
+ * The slots event must be before the metric events,
+ * because we only update the values of a topdown
+ * group once with the slots event.
+ */
+ if (!is_slots_event(leader)) {
+ for_each_sibling_event(sibling, leader) {
+ if (is_slots_event(sibling))
+ break;
+ if (is_metric_event(sibling))
+ return -EINVAL;
+ }
+ }
+ }
Thanks,
Kan