Support Lunar Lake and Arrow Lake core PMU
Subsystems (2)
v2 cover Support Lunar Lake and Arrow Lake core PMU
From: Kan Liang <[email protected]>
Changes since V1:
- Add x86/intel_pmu_max_num_pebs/counters/counters_fixed()
- Rename model-specific pebs_latency_data functions
- Rename V6 counter MSRs
From the core PMU' perspective, the Lunar Lake and Arrow Lake are the same, which are similar to the previous generation Meteor Lake. Both are hybrid platforms, with e-core and p-core.
The key differences include:
- The e-core supports 3 new fixed counters
- The p-core supports an updated PEBS Data Source format
- More GP counters (Updated event constraint table)
- New Architectural performance monitoring V6 (New Perfmon MSRs aliasing, umask2, eq).
- New PEBS format V6 (Counters Snapshotting group)
- New RDPMC metrics clear mode
The details for the above new features can be found in the Intel Architecture Instruction Set Extensions and Future Features (052).
https://cdrdv2.intel.com/v1/dl/getContent/671368
The counters may not be continuous anymore. Patch 1-2 converts the max number of counters to a mask of counters. The change is a generic change which impacts all X86 platforms.
Patch 3-4 supports all the legacy features on LNL and ARL.
Patch 5-7 supports the new Architectural performance monitoring V6.
Patch 8-11 supports the new PEBS format V6.
Patch 12 supports the new RDPMC metrics clear mode.
Kan Liang (13):
perf/x86/intel: Support the PEBS event mask
perf/x86: Support counter mask
perf/x86: Add Lunar Lake and Arrow Lake support perf/x86/intel: Rename model-specific pebs_latency_data functions perf/x86/intel: Support new data source for Lunar Lake perf/x86: Add config_mask to represent EVENTSEL bitmask perf/x86/intel: Support PERFEVTSEL extension perf/x86/intel: Support Perfmon MSRs aliasing perf/x86: Extend event update interface
perf: Extend perf_output_read
perf/x86/intel: Move PEBS event update after the sample output perf/x86/intel: Support PEBS counters snapshotting perf/x86/intel: Support RDPMC metrics clear mode
arch/x86/events/amd/core.c | 26 +- arch/x86/events/core.c | 123 +++---- arch/x86/events/intel/core.c | 471 ++++++++++++++++++++------- arch/x86/events/intel/ds.c | 288 +++++++++++++--- arch/x86/events/intel/knc.c | 2 +- arch/x86/events/intel/p4.c | 12 +- arch/x86/events/intel/p6.c | 2 +- arch/x86/events/perf_event.h | 105 +++++- arch/x86/events/perf_event_flags.h | 2 +- arch/x86/events/zhaoxin/core.c | 14 +- arch/x86/include/asm/intel_ds.h | 1 + arch/x86/include/asm/msr-index.h | 6 + arch/x86/include/asm/perf_event.h | 27 ++ include/uapi/linux/perf_event.h | 6 +- kernel/events/core.c | 15 +- tools/perf/Documentation/topdown.txt | 9 +- 16 files changed, 839 insertions(+), 270 deletions(-)
Signature · 1 line
2.35.1
PATCH v2 1/13 perf/x86/intel: Support the PEBS event mask
From: Kan Liang <[email protected]>
The current perf assumes that the counters that support PEBS are contiguous. But it's not guaranteed with the new leaf 0x23 introduced. The counters are enumerated with a counter mask. There may be holes in the counter mask for future platforms or in a virtualization environment.
Store the PEBS event mask rather than the maximum number of PEBS counters in the x86 PMU structures.
--- arch/x86/events/intel/core.c | 8 ++++---- arch/x86/events/intel/ds.c | 15 ++++++++------- arch/x86/events/perf_event.h | 14 ++++++++++++-- arch/x86/include/asm/intel_ds.h | 1 + 4 files changed, 25 insertions(+), 13 deletions(-)
arch/x86/events/intel/core.c +4 -4
@@ -4728,7 +4728,7 @@ static void intel_pmu_check_hybrid_pmus(struct x86_hybrid_pmu *pmu) {intel_pmu_check_num_counters (&pmu->num_counters, &pmu->num_counters_fixed, &pmu->intel_ctrl, (1ULL << pmu->num_counters_fixed) -1 ); - pmu->max_pebs_events =min_t (unsigned,MAX_PEBS_EVENTS , pmu->num_counters); + pmu->pebs_events_mask =intel_pmu_pebs_mask (GENMASK_ULL (pmu->num_counters -1 ,0 )); pmu->unconstrained = (struct event_constraint )__EVENT_CONSTRAINT (0 , (1ULL << pmu->num_counters) -1 ,0 , pmu->num_counters,0 ,0 );
@@ -6070,7 +6070,7 @@ static __always_inline int intel_pmu_init_hybrid(enum hybrid_pmu_type pmus) pmu->num_counters = x86_pmu.num_counters; pmu->num_counters_fixed = x86_pmu.num_counters_fixed; - pmu->max_pebs_events =min_t (unsigned,MAX_PEBS_EVENTS , pmu->num_counters); + pmu->pebs_events_mask =intel_pmu_pebs_mask (GENMASK_ULL (pmu->num_counters -1 ,0 )); pmu->unconstrained = (struct event_constraint )__EVENT_CONSTRAINT (0 , (1ULL << pmu->num_counters) -1 ,0 , pmu->num_counters,0 ,0 );
@@ -6193,7 +6193,7 @@ __init int intel_pmu_init(void) x86_pmu.events_maskl = ebx.full; x86_pmu.events_mask_len = eax.split.mask_length; - x86_pmu.max_pebs_events =min_t (unsigned,MAX_PEBS_EVENTS , x86_pmu.num_counters); + x86_pmu.pebs_events_mask =intel_pmu_pebs_mask (GENMASK_ULL (x86_pmu.num_counters -1 ,0 )); x86_pmu.pebs_capable =PEBS_COUNTER_MASK ;/*
@@ -6822,7 +6822,7 @@ __init int intel_pmu_init(void) pmu->num_counters_fixed = x86_pmu.num_counters_fixed; } - pmu->max_pebs_events =min_t (unsigned,MAX_PEBS_EVENTS , pmu->num_counters); + pmu->pebs_events_mask =intel_pmu_pebs_mask (GENMASK_ULL (pmu->num_counters -1 ,0 )); pmu->unconstrained = (struct event_constraint )__EVENT_CONSTRAINT (0 , (1ULL << pmu->num_counters) -1 ,0 , pmu->num_counters,0 ,0 );
arch/x86/events/intel/ds.c +8 -7
@@ -1137,7 +1137,7 @@ void intel_pmu_pebs_sched_task(struct perf_event_pmu_context *pmu_ctx, bool schestatic inline void pebs_update_threshold (struct cpu_hw_events *cpuc) {struct debug_store *ds = cpuc->ds; -int max_pebs_events =hybrid (cpuc->pmu, max_pebs_events); +int max_pebs_events =intel_pmu_max_num_pebs (cpuc->pmu);int num_counters_fixed =hybrid (cpuc->pmu, num_counters_fixed);u64 threshold;int reserved;
@@ -2157,6 +2157,7 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs, struct perf_sample_dvoid *base, *at, *top;short counts[INTEL_PMC_IDX_FIXED +MAX_FIXED_PEBS_EVENTS ] = {};short error[INTEL_PMC_IDX_FIXED +MAX_FIXED_PEBS_EVENTS ] = {}; +int max_pebs_events =intel_pmu_max_num_pebs (NULL );int bit, i, size;u64 mask;
@@ -2168,8 +2169,8 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs, struct perf_sample_d ds->pebs_index = ds->pebs_buffer_base; - mask = (1ULL << x86_pmu.max_pebs_events) -1 ; - size = x86_pmu.max_pebs_events; + mask = x86_pmu.pebs_events_mask; + size = max_pebs_events;if (x86_pmu.flags &PMU_FL_PEBS_ALL ) { mask |= ((1ULL << x86_pmu.num_counters_fixed) -1 ) <<INTEL_PMC_IDX_FIXED ; size =INTEL_PMC_IDX_FIXED + x86_pmu.num_counters_fixed;
@@ -2208,8 +2209,9 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs, struct perf_sample_d pebs_status = p->status = cpuc->pebs_enabled; bit =find_first_bit ((unsigned long *)&pebs_status, - x86_pmu.max_pebs_events); -if (bit >= x86_pmu.max_pebs_events) + max_pebs_events); + +if (!(x86_pmu.pebs_events_mask & (1 << bit)))continue ;/*
@@ -2267,7 +2269,6 @@ static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_d {short counts[INTEL_PMC_IDX_FIXED +MAX_FIXED_PEBS_EVENTS ] = {};struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); -int max_pebs_events =hybrid (cpuc->pmu, max_pebs_events);int num_counters_fixed =hybrid (cpuc->pmu, num_counters_fixed);struct debug_store *ds = cpuc->ds;struct perf_event *event;
@@ -2283,7 +2284,7 @@ static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_d ds->pebs_index = ds->pebs_buffer_base; - mask = ((1ULL << max_pebs_events) -1 ) | + mask =hybrid (cpuc->pmu, pebs_events_mask) | (((1ULL << num_counters_fixed) -1 ) <<INTEL_PMC_IDX_FIXED ); size =INTEL_PMC_IDX_FIXED + num_counters_fixed;
arch/x86/events/perf_event.h +12 -2
@@ -684,7 +684,7 @@ struct x86_hybrid_pmu {cpumask_t supported_cpus;union perf_capabilities intel_cap;u64 intel_ctrl; -int max_pebs_events; +u64 pebs_events_mask;int num_counters;int num_counters_fixed;struct event_constraint unconstrained;
@@ -852,7 +852,7 @@ struct x86_pmu { pebs_ept :1 ;int pebs_record_size;int pebs_buffer_size; -int max_pebs_events; +u64 pebs_events_mask;void (*drain_pebs)(struct pt_regs *regs,struct perf_sample_data *data);struct event_constraint *pebs_constraints;void (*pebs_aliases)(struct perf_event *event);
@@ -1661,6 +1661,16 @@ static inline int is_ht_workaround_enabled(void) return !!(x86_pmu.flags & PMU_FL_EXCL_ENABLED); } +static inline u64 intel_pmu_pebs_mask (u64 cntr_mask) +{ +return MAX_PEBS_EVENTS_MASK & cntr_mask; +} + +static inline int intel_pmu_max_num_pebs (struct pmu *pmu) +{ +return find_last_bit ((unsigned long *)&hybrid (pmu, pebs_events_mask),MAX_PEBS_EVENTS ) +1 ; +} +#else /* CONFIG_CPU_SUP_INTEL */ static inline void reserve_ds_buffers (void )
arch/x86/include/asm/intel_ds.h +1 -0
@@ -9,6 +9,7 @@/* The maximal number of PEBS events: */ #define MAX_PEBS_EVENTS_FMT4 8#define MAX_PEBS_EVENTS 32 +#define MAX_PEBS_EVENTS_MASK GENMASK_ULL(MAX_PEBS_EVENTS - 1, 0)#define MAX_FIXED_PEBS_EVENTS 16/*
Signature · 1 line
2.35.1
PATCH v2 3/13 perf/x86: Add Lunar Lake and Arrow Lake support
In reply to [email protected]From: Kan Liang <[email protected]>
From PMU's perspective, Lunar Lake and Arrow Lake are similar to the previous generation Meteor Lake. Both are hybrid platforms, with e-core and p-core.
The key differences include:
- The e-core supports 3 new fixed counters
- The p-core supports an updated PEBS Data Source format
- More GP counters (Updated event constraint table)
- New Architectural performance monitoring V6 (New Perfmon MSRs aliasing, umask2, eq).
- New PEBS format V6 (Counters Snapshotting group)
- New RDPMC metrics clear mode
The legacy features, the 3 new fixed counters and updated event constraint table are enabled in this patch.
The new PEBS data source format, the architectural performance monitoring V6, the PEBS format V6, and the new RDPMC metrics clear mode are supported in the following patches.
--- arch/x86/events/intel/core.c | 117 ++++++++++++++++++++++++++++++ arch/x86/events/intel/ds.c | 24 ++++++ arch/x86/events/perf_event.h | 2 + arch/x86/include/asm/perf_event.h | 4 + 4 files changed, 147 insertions(+)
arch/x86/events/intel/core.c +117 -0
@@ -220,6 +220,17 @@ static struct event_constraint intel_grt_event_constraints[] __read_mostly = {EVENT_CONSTRAINT_END }; +static struct event_constraint intel_skt_event_constraints[]__read_mostly = { +FIXED_EVENT_CONSTRAINT (0x00c0 ,0 ),/* INST_RETIRED.ANY */ +FIXED_EVENT_CONSTRAINT (0x003c ,1 ),/* CPU_CLK_UNHALTED.CORE */ +FIXED_EVENT_CONSTRAINT (0x0300 ,2 ),/* pseudo CPU_CLK_UNHALTED.REF */ +FIXED_EVENT_CONSTRAINT (0x013c ,2 ),/* CPU_CLK_UNHALTED.REF_TSC_P */ +FIXED_EVENT_CONSTRAINT (0x0073 ,4 ),/* TOPDOWN_BAD_SPECULATION.ALL */ +FIXED_EVENT_CONSTRAINT (0x019c ,5 ),/* TOPDOWN_FE_BOUND.ALL */ +FIXED_EVENT_CONSTRAINT (0x02c2 ,6 ),/* TOPDOWN_RETIRING.ALL */ +EVENT_CONSTRAINT_END +}; +static struct event_constraint intel_skl_event_constraints[] = {FIXED_EVENT_CONSTRAINT (0x00c0 ,0 ),/* INST_RETIRED.ANY */ FIXED_EVENT_CONSTRAINT (0x003c ,1 ),/* CPU_CLK_UNHALTED.CORE */
@@ -370,6 +381,55 @@ static struct extra_reg intel_rwc_extra_regs[] __read_mostly = {EVENT_EXTRA_END }; +static struct event_constraint intel_lnc_event_constraints[] = { +FIXED_EVENT_CONSTRAINT (0x00c0 ,0 ),/* INST_RETIRED.ANY */ +FIXED_EVENT_CONSTRAINT (0x0100 ,0 ),/* INST_RETIRED.PREC_DIST */ +FIXED_EVENT_CONSTRAINT (0x003c ,1 ),/* CPU_CLK_UNHALTED.CORE */ +FIXED_EVENT_CONSTRAINT (0x0300 ,2 ),/* CPU_CLK_UNHALTED.REF */ +FIXED_EVENT_CONSTRAINT (0x013c ,2 ),/* CPU_CLK_UNHALTED.REF_TSC_P */ +FIXED_EVENT_CONSTRAINT (0x0400 ,3 ),/* SLOTS */ +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_RETIRING ,0 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_BAD_SPEC ,1 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_FE_BOUND ,2 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_BE_BOUND ,3 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_HEAVY_OPS ,4 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_BR_MISPREDICT ,5 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_FETCH_LAT ,6 ), +METRIC_EVENT_CONSTRAINT (INTEL_TD_METRIC_MEM_BOUND ,7 ), + +INTEL_UEVENT_CONSTRAINT (0x0148 ,0x4 ), +INTEL_UEVENT_CONSTRAINT (0x0175 ,0x4 ), + +INTEL_EVENT_CONSTRAINT (0x2e ,0x3ff ), +INTEL_EVENT_CONSTRAINT (0x3c ,0x3ff ), +/* +* Generally event codes < 0x90 are restricted to counters 0-3. +* The 0x2E and 0x3C are exception, which has no restriction. +*/ +INTEL_EVENT_CONSTRAINT_RANGE (0x01 ,0x8f ,0xf ), + +INTEL_UEVENT_CONSTRAINT (0x01a3 ,0xf ), +INTEL_UEVENT_CONSTRAINT (0x02a3 ,0xf ), +INTEL_UEVENT_CONSTRAINT (0x08a3 ,0x4 ), +INTEL_UEVENT_CONSTRAINT (0x0ca3 ,0x4 ), +INTEL_UEVENT_CONSTRAINT (0x04a4 ,0x1 ), +INTEL_UEVENT_CONSTRAINT (0x08a4 ,0x1 ), +INTEL_UEVENT_CONSTRAINT (0x10a4 ,0x1 ), +INTEL_UEVENT_CONSTRAINT (0x01b1 ,0x8 ), +INTEL_UEVENT_CONSTRAINT (0x02cd ,0x3 ), +INTEL_EVENT_CONSTRAINT (0xce ,0x1 ), + +INTEL_EVENT_CONSTRAINT_RANGE (0xd0 ,0xdf ,0xf ), +/* +* Generally event codes >= 0x90 are likely to have no restrictions. +* The exception are defined as above. +*/ +INTEL_EVENT_CONSTRAINT_RANGE (0x90 ,0xfe ,0x3ff ), + +EVENT_CONSTRAINT_END +}; + +EVENT_ATTR_STR (mem-loads, mem_ld_nhm,"event=0x0b,umask=0x10,ldlat=3" );EVENT_ATTR_STR (mem-loads, mem_ld_snb,"event=0xcd,umask=0x1,ldlat=3" );EVENT_ATTR_STR (mem-stores, mem_st_snb,"event=0xcd,umask=0x2" );
@@ -5790,6 +5850,23 @@ static struct attribute *adl_hybrid_events_attrs[] = { NULL, }; +EVENT_ATTR_STR_HYBRID (topdown-retiring, td_retiring_lnl,"event=0xc2,umask=0x02;event=0x00,umask=0x80" , hybrid_big_small); +EVENT_ATTR_STR_HYBRID (topdown-fe-bound, td_fe_bound_lnl,"event=0x9c,umask=0x01;event=0x00,umask=0x82" , hybrid_big_small); +EVENT_ATTR_STR_HYBRID (topdown-be-bound, td_be_bound_lnl,"event=0xa4,umask=0x02;event=0x00,umask=0x83" , hybrid_big_small); + +static struct attribute *lnl_hybrid_events_attrs[] = { +EVENT_PTR (slots_adl), +EVENT_PTR (td_retiring_lnl), +EVENT_PTR (td_bad_spec_adl), +EVENT_PTR (td_fe_bound_lnl), +EVENT_PTR (td_be_bound_lnl), +EVENT_PTR (td_heavy_ops_adl), +EVENT_PTR (td_br_mis_adl), +EVENT_PTR (td_fetch_lat_adl), +EVENT_PTR (td_mem_bound_adl), +NULL +}; +/* Must be in IDX order */ EVENT_ATTR_STR_HYBRID (mem-loads, mem_ld_adl,"event=0xd0,umask=0x5,ldlat=3;event=0xcd,umask=0x1,ldlat=3" , hybrid_big_small);EVENT_ATTR_STR_HYBRID (mem-stores, mem_st_adl,"event=0xd0,umask=0x6;event=0xcd,umask=0x2" , hybrid_big_small);
@@ -6139,6 +6216,21 @@ static __always_inline void intel_pmu_init_grt(struct pmu *pmu)intel_pmu_ref_cycles_ext (); } +static __always_inline void intel_pmu_init_lnc(struct pmu *pmu) +{ +intel_pmu_init_glc (pmu); +hybrid (pmu, event_constraints) = intel_lnc_event_constraints; +hybrid (pmu, pebs_constraints) = intel_lnc_pebs_event_constraints; +hybrid (pmu, extra_regs) = intel_rwc_extra_regs; +} + +static __always_inline void intel_pmu_init_skt(struct pmu *pmu) +{ +intel_pmu_init_grt (pmu); +hybrid (pmu, event_constraints) = intel_skt_event_constraints; +hybrid (pmu, extra_regs) = intel_cmt_extra_regs; +} +__init int intel_pmu_init (void ) {struct attribute **extra_skl_attr = &empty_attrs;
@@ -6864,6 +6956,31 @@ __init int intel_pmu_init(void) name ="meteorlake_hybrid" ;break ; +case INTEL_FAM6_LUNARLAKE_M : +case INTEL_FAM6_ARROWLAKE : +intel_pmu_init_hybrid (hybrid_big_small); + + x86_pmu.get_event_constraints = mtl_get_event_constraints; + x86_pmu.hw_config = adl_hw_config; + + td_attr = lnl_hybrid_events_attrs; + mem_attr = mtl_hybrid_mem_attrs; + tsx_attr = adl_hybrid_tsx_attrs; + extra_attr =boot_cpu_has (X86_FEATURE_RTM ) ? + mtl_hybrid_extra_attr_rtm : mtl_hybrid_extra_attr; + +/* Initialize big core specific PerfMon capabilities.*/ + pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX ]; +intel_pmu_init_lnc (&pmu->pmu); + +/* Initialize Atom core specific PerfMon capabilities.*/ + pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_ATOM_IDX ]; +intel_pmu_init_skt (&pmu->pmu); + +pr_cont ("Lunarlake Hybrid events, " ); + name ="lunarlake_hybrid" ; +break ; +default :switch (x86_pmu.version) {case 1 :
arch/x86/events/intel/ds.c +24 -0
@@ -1086,6 +1086,30 @@ struct event_constraint intel_glc_pebs_event_constraints[] = { EVENT_CONSTRAINT_END }; +struct event_constraint intel_lnc_pebs_event_constraints[] = { +INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ +INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), + +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x11d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x12d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_STORES */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x21d0 ,0xf ),/* MEM_INST_RETIRED.LOCK_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x41d0 ,0xf ),/* MEM_INST_RETIRED.SPLIT_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x42d0 ,0xf ),/* MEM_INST_RETIRED.SPLIT_STORES */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x81d0 ,0xf ),/* MEM_INST_RETIRED.ALL_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x82d0 ,0xf ),/* MEM_INST_RETIRED.ALL_STORES */ + +INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD_RANGE (0xd1 ,0xd4 ,0xf ), + +INTEL_FLAGS_EVENT_CONSTRAINT (0xd0 ,0xf ), + +/* +* Everything else is handled by PMU_FL_PEBS_ALL, because we +* need the full constraints from the main table. +*/ + +EVENT_CONSTRAINT_END +}; +struct event_constraint *intel_pebs_constraints (struct perf_event *event) {struct event_constraint *pebs_constraints =hybrid (event->pmu, pebs_constraints);
arch/x86/events/perf_event.h +2 -0
@@ -1582,6 +1582,8 @@ extern struct event_constraint intel_icl_pebs_event_constraints[];extern struct event_constraint intel_glc_pebs_event_constraints[]; +extern struct event_constraint intel_lnc_pebs_event_constraints[]; +struct event_constraint *intel_pebs_constraints (struct perf_event *event);void intel_pmu_pebs_add (struct perf_event *event);
arch/x86/include/asm/perf_event.h +4 -0
@@ -307,6 +307,10 @@ struct x86_pmu_capability {#define INTEL_PMC_IDX_FIXED_SLOTS (INTEL_PMC_IDX_FIXED + 3)#define INTEL_PMC_MSK_FIXED_SLOTS (1ULL << INTEL_PMC_IDX_FIXED_SLOTS) +/* TOPDOWN_BAD_SPECULATION.ALL: fixed counter 4 (Atom only) */ +/* TOPDOWN_FE_BOUND.ALL: fixed counter 5 (Atom only) */ +/* TOPDOWN_RETIRING.ALL: fixed counter 6 (Atom only) */ +static inline bool use_fixed_pseudo_encoding (u64 code) {return !(code &0xff );
Signature · 1 line
2.35.1
PATCH v2 7/13 perf/x86/intel: Support PERFEVTSEL extension
In reply to [email protected]From: Kan Liang <[email protected]>
Two new fields (the unit mask2, and the equal flag) are added in the IA32_PERFEVTSELx MSRs. They can be enumerated by the CPUID.23H.0.EBX.
Update the config_mask in x86_pmu and x86_hybrid_pmu for the true layout of the PERFEVTSEL.
Expose the new formats into sysfs if they are available. The umask extension reuses the same format attr name "umask" as the previous umask. Add umask2_show to determine/display the correct format for the current machine.
--- arch/x86/events/intel/core.c | 69 +++++++++++++++++++++++++++++-- arch/x86/include/asm/perf_event.h | 4 ++ 2 files changed, 69 insertions(+), 4 deletions(-)
arch/x86/events/intel/core.c +65 -4
@@ -4632,8 +4632,55 @@ PMU_FORMAT_ATTR(pc, "config:19" );PMU_FORMAT_ATTR (any,"config:21" );/* v3 + */ PMU_FORMAT_ATTR (inv,"config:23" );PMU_FORMAT_ATTR (cmask,"config:24-31" ); -PMU_FORMAT_ATTR (in_tx,"config:32" ); -PMU_FORMAT_ATTR (in_tx_cp,"config:33" ); +PMU_FORMAT_ATTR (in_tx,"config:32" ); +PMU_FORMAT_ATTR (in_tx_cp,"config:33" ); +PMU_FORMAT_ATTR (eq,"config:36" );/* v6 + */ + +static ssize_t umask2_show (struct device *dev, +struct device_attribute *attr, +char *page) +{ +u64 mask =hybrid (dev_get_drvdata (dev), config_mask) &ARCH_PERFMON_EVENTSEL_UMASK2 ; + +if (mask ==ARCH_PERFMON_EVENTSEL_UMASK2 ) +return sprintf (page,"config:8-15,40-47\n" ); + +/* Roll back to the old format if umask2 is not supported. */ +return sprintf (page,"config:8-15\n" ); +} + +static struct device_attribute format_attr_umask2 = +__ATTR (umask,0444 , umask2_show,NULL ); + +static struct attribute *format_evtsel_ext_attrs[] = { + &format_attr_umask2.attr, + &format_attr_eq.attr, +NULL +}; + +static umode_t +evtsel_ext_is_visible (struct kobject *kobj,struct attribute *attr,int i) +{ +struct device *dev =kobj_to_dev (kobj); +u64 mask; + +/* +* The umask and umask2 have different formats but share the +* same attr name. In update mode, the previous value of the +* umask is unconditionally removed before is_visible. If +* umask2 format is not enumerated, it's impossible to roll +* back to the old format. +* Does the check in umask2_show rather than is_visible. +*/ +if (i ==0 ) +return attr->mode; + + mask =hybrid (dev_get_drvdata (dev), config_mask); +if (i ==1 ) +return (mask &ARCH_PERFMON_EVENTSEL_EQ ) ? attr->mode :0 ; + +return 0 ; +}static struct attribute *intel_arch_formats_attr[] = { &format_attr_event.attr,
@@ -4786,8 +4833,14 @@ static inline bool intel_pmu_broken_perf_cap(void)static void update_pmu_cap (struct x86_hybrid_pmu *pmu) { -unsigned int sub_bitmaps =cpuid_eax (ARCH_PERFMON_EXT_LEAF ); -unsigned int eax, ebx, ecx, edx; +unsigned int sub_bitmaps, eax, ebx, ecx, edx; + +cpuid (ARCH_PERFMON_EXT_LEAF , &sub_bitmaps, &ebx, &ecx, &edx); + +if (ebx &ARCH_PERFMON_EXT_UMASK2 ) + pmu->config_mask |=ARCH_PERFMON_EVENTSEL_UMASK2 ; +if (ebx &ARCH_PERFMON_EXT_EQ ) + pmu->config_mask |=ARCH_PERFMON_EVENTSEL_EQ ;if (sub_bitmaps &ARCH_PERFMON_NUM_COUNTER_LEAF_BIT ) { cpuid_count(ARCH_PERFMON_EXT_LEAF, ARCH_PERFMON_NUM_COUNTER_LEAF,
@@ -5810,6 +5863,12 @@ static struct attribute_group group_format_extra_skl = { .is_visible = exra_is_visible, }; +static struct attribute_group group_format_evtsel_ext = { + .name ="format" , + .attrs = format_evtsel_ext_attrs, + .is_visible = evtsel_ext_is_visible, +}; +static struct attribute_group group_default = { .attrs = intel_pmu_attrs, .is_visible = default_is_visible,
@@ -5823,6 +5882,7 @@ static const struct attribute_group *attr_update[] = { &group_caps_lbr, &group_format_extra, &group_format_extra_skl, + &group_format_evtsel_ext, &group_default, NULL, };
@@ -6042,6 +6102,7 @@ static const struct attribute_group *hybrid_attr_update[] = { &group_caps_gen, &group_caps_lbr, &hybrid_group_format_extra, + &group_format_evtsel_ext, &group_default, &hybrid_group_cpus, NULL,
arch/x86/include/asm/perf_event.h +4 -0
@@ -32,6 +32,8 @@#define ARCH_PERFMON_EVENTSEL_INV (1ULL << 23)#define ARCH_PERFMON_EVENTSEL_CMASK 0xFF000000ULL#define ARCH_PERFMON_EVENTSEL_BR_CNTR (1ULL << 35) +#define ARCH_PERFMON_EVENTSEL_EQ (1ULL << 36) +#define ARCH_PERFMON_EVENTSEL_UMASK2 (0xFFULL << 40)#define INTEL_FIXED_BITS_MASK 0xFULL#define INTEL_FIXED_BITS_STRIDE 4
@@ -185,6 +187,8 @@ union cpuid10_edx {* detection/enumeration details: */ #define ARCH_PERFMON_EXT_LEAF 0x00000023 +#define ARCH_PERFMON_EXT_UMASK2 0x1 +#define ARCH_PERFMON_EXT_EQ 0x2#define ARCH_PERFMON_NUM_COUNTER_LEAF_BIT 0x1#define ARCH_PERFMON_NUM_COUNTER_LEAF 0x1
Signature · 1 line
2.35.1
PATCH v2 4/13 perf/x86/intel: Rename model-specific pebs_latency_data functions
In reply to [email protected]From: Kan Liang <[email protected]>
The model-specific pebs_latency_data functions of ADL and MTL use the "small" as a postfix to indicate the e-core. The postfix is too generic for a model-specific function. It cannot provide useful information that can directly map it to a specific uarch, which can facilitate the development and maintenance.
Use the abbr of the uarch to rename the model-specific functions.
--- arch/x86/events/intel/core.c | 8 ++++---- arch/x86/events/intel/ds.c | 20 ++++++++++---------- arch/x86/events/perf_event.h | 4 ++-- 3 files changed, 16 insertions(+), 16 deletions(-)
arch/x86/events/intel/core.c +4 -4
@@ -6509,7 +6509,7 @@ __init int intel_pmu_init(void)case INTEL_FAM6_ATOM_GRACEMONT :intel_pmu_init_grt (NULL );intel_pmu_pebs_data_source_grt (); - x86_pmu.pebs_latency_data = adl_latency_data_small; + x86_pmu.pebs_latency_data = adl_latency_data_grt; x86_pmu.get_event_constraints = tnt_get_event_constraints; td_attr = tnt_events_attrs; mem_attr = grt_mem_attrs;
@@ -6523,7 +6523,7 @@ __init int intel_pmu_init(void)intel_pmu_init_grt (NULL ); x86_pmu.extra_regs = intel_cmt_extra_regs;intel_pmu_pebs_data_source_cmt (); - x86_pmu.pebs_latency_data = mtl_latency_data_small; + x86_pmu.pebs_latency_data = mtl_latency_data_cmt; x86_pmu.get_event_constraints = cmt_get_event_constraints; td_attr = cmt_events_attrs; mem_attr = grt_mem_attrs;
@@ -6874,7 +6874,7 @@ __init int intel_pmu_init(void)*/ intel_pmu_init_hybrid (hybrid_big_small); - x86_pmu.pebs_latency_data = adl_latency_data_small; + x86_pmu.pebs_latency_data = adl_latency_data_grt; x86_pmu.get_event_constraints = adl_get_event_constraints; x86_pmu.hw_config = adl_hw_config; x86_pmu.get_hybrid_cpu_type = adl_get_hybrid_cpu_type;
@@ -6931,7 +6931,7 @@ __init int intel_pmu_init(void)case INTEL_FAM6_METEORLAKE_L :intel_pmu_init_hybrid (hybrid_big_small); - x86_pmu.pebs_latency_data = mtl_latency_data_small; + x86_pmu.pebs_latency_data = mtl_latency_data_cmt; x86_pmu.get_event_constraints = mtl_get_event_constraints; x86_pmu.hw_config = adl_hw_config;
arch/x86/events/intel/ds.c +10 -10
@@ -257,8 +257,8 @@ static inline void pebs_set_tlb_lock(u64 *val, bool tlb, bool lock) }/* Retrieve the latency data for e-core of ADL */ -static u64 __adl_latency_data_small (struct perf_event *event,u64 status, -u8 dse,bool tlb,bool lock,bool blk) +static u64 __adl_latency_data_grt (struct perf_event *event,u64 status, +u8 dse,bool tlb,bool lock,bool blk) {u64 val;
@@ -277,27 +277,27 @@ static u64 __adl_latency_data_small(struct perf_event *event, u64 status,return val; } -u64 adl_latency_data_small (struct perf_event *event,u64 status) +u64 adl_latency_data_grt (struct perf_event *event,u64 status) {union intel_x86_pebs_dse dse; dse.val = status; -return __adl_latency_data_small (event, status, dse.ld_dse, - dse.ld_locked, dse.ld_stlb_miss, - dse.ld_data_blk); +return __adl_latency_data_grt (event, status, dse.ld_dse, + dse.ld_locked, dse.ld_stlb_miss, + dse.ld_data_blk); }/* Retrieve the latency data for e-core of MTL */ -u64 mtl_latency_data_small (struct perf_event *event,u64 status) +u64 mtl_latency_data_cmt (struct perf_event *event,u64 status) {union intel_x86_pebs_dse dse; dse.val = status; -return __adl_latency_data_small (event, status, dse.mtl_dse, - dse.mtl_stlb_miss, dse.mtl_locked, - dse.mtl_fwd_blk); +return __adl_latency_data_grt (event, status, dse.mtl_dse, + dse.mtl_stlb_miss, dse.mtl_locked, + dse.mtl_fwd_blk); }static u64 load_latency_data (struct perf_event *event,u64 status)
arch/x86/events/perf_event.h +2 -2
@@ -1548,9 +1548,9 @@ void intel_pmu_disable_bts(void);int intel_pmu_drain_bts_buffer (void ); -u64 adl_latency_data_small (struct perf_event *event,u64 status); +u64 adl_latency_data_grt (struct perf_event *event,u64 status); -u64 mtl_latency_data_small (struct perf_event *event,u64 status); +u64 mtl_latency_data_cmt (struct perf_event *event,u64 status);extern struct event_constraint intel_core2_pebs_event_constraints[];
Signature · 1 line
2.35.1
PATCH v2 6/13 perf/x86: Add config_mask to represent EVENTSEL bitmask
In reply to [email protected]From: Kan Liang <[email protected]>
Different vendors may support different fields in EVENTSEL MSR, such as Intel would introduce new fields umask2 and eq bits in EVENTSEL MSR since Perfmon version 6. However, a fixed mask X86_RAW_EVENT_MASK is used to filter the attr.config.
Introduce a new config_mask to record the real supported EVENTSEL bitmask.
Only apply it to the existing code now. No functional change.
--- arch/x86/events/core.c | 5 ++++- arch/x86/events/intel/core.c | 1 + arch/x86/events/perf_event.h | 7 +++++++ 3 files changed, 12 insertions(+), 1 deletion(-)
arch/x86/events/core.c +4 -1
@@ -624,7 +624,7 @@ int x86_pmu_hw_config(struct perf_event *event) event->hw.config |=ARCH_PERFMON_EVENTSEL_OS ;if (event->attr.type == event->pmu->type) - event->hw.config |= event->attr.config &X86_RAW_EVENT_MASK ; + event->hw.config |=x86_pmu_get_event_config (event);if (event->attr.sample_period && x86_pmu.limit_period) {s64 left = event->attr.sample_period;
@@ -2098,6 +2098,9 @@ static int __init init_hw_perf_events(void)if (!x86_pmu.intel_ctrl) x86_pmu.intel_ctrl = x86_pmu.cntr_mask64; +if (!x86_pmu.config_mask) + x86_pmu.config_mask =X86_RAW_EVENT_MASK ; +perf_events_lapic_init ();register_nmi_handler (NMI_LOCAL , perf_event_nmi_handler,0 ,"PMI" );
arch/x86/events/intel/core.c +1 -0
@@ -6144,6 +6144,7 @@ static __always_inline int intel_pmu_init_hybrid(enum hybrid_pmu_type pmus) pmu->cntr_mask64 = x86_pmu.cntr_mask64; pmu->fixed_cntr_mask64 = x86_pmu.fixed_cntr_mask64; pmu->pebs_events_mask =intel_pmu_pebs_mask (pmu->cntr_mask64); + pmu->config_mask =X86_RAW_EVENT_MASK ; pmu->unconstrained = (struct event_constraint )__EVENT_CONSTRAINT (0 , pmu->cntr_mask64,0 ,x86_pmu_num_counters (&pmu->pmu),0 ,0 );
arch/x86/events/perf_event.h +7 -0
@@ -695,6 +695,7 @@ struct x86_hybrid_pmu {union perf_capabilities intel_cap;u64 intel_ctrl;u64 pebs_events_mask; +u64 config_mask;union {u64 cntr_mask64;unsigned long cntr_mask[BITS_TO_LONGS (X86_PMC_IDX_MAX )];
@@ -790,6 +791,7 @@ struct x86_pmu {int (*rdpmc_index)(int index);u64 (*event_map)(int);int max_events; +u64 config_mask;union {u64 cntr_mask64;unsigned long cntr_mask[BITS_TO_LONGS (X86_PMC_IDX_MAX )];
@@ -1241,6 +1243,11 @@ static inline int x86_pmu_max_num_counters_fixed(struct pmu *pmu)return find_last_bit (hybrid (pmu, fixed_cntr_mask),INTEL_PMC_MAX_FIXED ) +1 ; } +static inline u64 x86_pmu_get_event_config (struct perf_event *event) +{ +return event->attr.config &hybrid (event->pmu, config_mask); +} +extern struct event_constraint emptyconstraint;extern struct event_constraint unconstrained;
Signature · 1 line
2.35.1
PATCH v2 5/13 perf/x86/intel: Support new data source for Lunar Lake
In reply to [email protected]From: Kan Liang <[email protected]>
A new PEBS data source format is introduced for the p-core of Lunar Lake. The data source field is extended to 8 bits with new encodings.
A new layout is introduced into the union intel_x86_pebs_dse. Introduce the lnl_latency_data() to parse the new format. Enlarge the pebs_data_source[] accordingly to include new encodings.
Only the mem load and the mem store events can generate the data source. Introduce INTEL_HYBRID_LDLAT_CONSTRAINT and INTEL_HYBRID_STLAT_CONSTRAINT to mark them.
Add two new bits for the new cache-related data src, L2_MHB and MSC. The L2_MHB is short for L2 Miss Handling Buffer, which is similar to LFB (Line Fill Buffer), but to track the L2 Cache misses. The MSC stands for the memory-side cache.
--- arch/x86/events/intel/core.c | 2 + arch/x86/events/intel/ds.c | 88 ++++++++++++++++++++++++++++++++- arch/x86/events/perf_event.h | 16 +++++- include/uapi/linux/perf_event.h | 6 ++- 4 files changed, 107 insertions(+), 5 deletions(-)
arch/x86/events/intel/core.c +2 -0
@@ -6960,6 +6960,7 @@ __init int intel_pmu_init(void)case INTEL_FAM6_ARROWLAKE :intel_pmu_init_hybrid (hybrid_big_small); + x86_pmu.pebs_latency_data = lnl_latency_data; x86_pmu.get_event_constraints = mtl_get_event_constraints; x86_pmu.hw_config = adl_hw_config;
@@ -6977,6 +6978,7 @@ __init int intel_pmu_init(void) pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_ATOM_IDX ];intel_pmu_init_skt (&pmu->pmu); +intel_pmu_pebs_data_source_lnl ();pr_cont ("Lunarlake Hybrid events, " ); name ="lunarlake_hybrid" ;break ;
arch/x86/events/intel/ds.c +86 -2
124 lines not on this page · raw message
arch/x86/events/perf_event.h +15 -1
@@ -476,6 +476,14 @@ struct cpu_hw_events {__EVENT_CONSTRAINT (c, n,INTEL_ARCH_EVENT_MASK |X86_ALL_EVENT_FLAGS , \ HWEIGHT(n), 0, PERF_X86_EVENT_PEBS_LAT_HYBRID) +#define INTEL_HYBRID_LDLAT_CONSTRAINT (c, n) \ + __EVENT_CONSTRAINT(c, n, INTEL_ARCH_EVENT_MASK|X86_ALL_EVENT_FLAGS, \ + HWEIGHT(n), 0, PERF_X86_EVENT_PEBS_LAT_HYBRID|PERF_X86_EVENT_PEBS_LD_HSW) + +#define INTEL_HYBRID_STLAT_CONSTRAINT (c, n) \ + __EVENT_CONSTRAINT(c, n, INTEL_ARCH_EVENT_MASK|X86_ALL_EVENT_FLAGS, \ + HWEIGHT(n), 0, PERF_X86_EVENT_PEBS_LAT_HYBRID|PERF_X86_EVENT_PEBS_ST_HSW) +/* Event constraint, but match on all event flags too. */ #define INTEL_FLAGS_EVENT_CONSTRAINT (c, n) \ EVENT_CONSTRAINT(c, n, ARCH_PERFMON_EVENTSEL_EVENT|X86_ALL_EVENT_FLAGS)
@@ -655,8 +663,10 @@ enum { x86_lbr_exclusive_max, }; -#define PERF_PEBS_DATA_SOURCE_MAX 0x10 +#define PERF_PEBS_DATA_SOURCE_MAX 0x100#define PERF_PEBS_DATA_SOURCE_MASK (PERF_PEBS_DATA_SOURCE_MAX - 1) +#define PERF_PEBS_DATA_SOURCE_GRT_MAX 0x10 +#define PERF_PEBS_DATA_SOURCE_GRT_MASK (PERF_PEBS_DATA_SOURCE_GRT_MAX - 1)enum hybrid_cpu_type {HYBRID_INTEL_NONE ,
@@ -1552,6 +1562,8 @@ u64 adl_latency_data_grt(struct perf_event *event, u64 status);u64 mtl_latency_data_cmt (struct perf_event *event,u64 status); +u64 lnl_latency_data (struct perf_event *event,u64 status); +extern struct event_constraint intel_core2_pebs_event_constraints[];extern struct event_constraint intel_atom_pebs_event_constraints[];
@@ -1673,6 +1685,8 @@ void intel_pmu_pebs_data_source_mtl(void);void intel_pmu_pebs_data_source_cmt (void ); +void intel_pmu_pebs_data_source_lnl (void ); +int intel_pmu_setup_lbr_filter (struct perf_event *event);void intel_pt_interrupt (void );
include/uapi/linux/perf_event.h +4 -2
@@ -1349,12 +1349,14 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0x7 available */ +#define PERF_MEM_LVLNUM_L2_MHB 0x05/* L2 Miss Handling Buffer */ +#define PERF_MEM_LVLNUM_MSC 0x06/* Memory-side Cache */ +/* 0x7 available */ #define PERF_MEM_LVLNUM_UNC 0x08/* Uncached */ #define PERF_MEM_LVLNUM_CXL 0x09/* CXL */ #define PERF_MEM_LVLNUM_IO 0x0a/* I/O */ #define PERF_MEM_LVLNUM_ANY_CACHE 0x0b/* Any cache */ -#define PERF_MEM_LVLNUM_LFB 0x0c/* LFB */ +#define PERF_MEM_LVLNUM_LFB 0x0c/* LFB / L1 Miss Handling Buffer */ #define PERF_MEM_LVLNUM_RAM 0x0d/* RAM */ #define PERF_MEM_LVLNUM_PMEM 0x0e/* PMEM */ #define PERF_MEM_LVLNUM_NA 0x0f/* N/A */
Signature · 1 line
2.35.1
PATCH v2 8/13 perf/x86/intel: Support Perfmon MSRs aliasing
In reply to [email protected]From: Kan Liang <[email protected]>
The architectural performance monitoring V6 supports a new range of counters' MSRs in the 19xxH address range. They include all the GP counter MSRs, the GP control MSRs, and the fixed counter MSRs.
The step between each sibling counter is 4. Add intel_pmu_addr_offset() to calculate the correct offset.
Add fixedctr in struct x86_pmu to store the address of the fixed counter
0. It can be used to calculate the rest of the fixed counters.
The MSR address of the fixed counter control is not changed.
--- arch/x86/events/core.c | 7 +++---- arch/x86/events/intel/core.c | 17 ++++++++++++++++- arch/x86/events/perf_event.h | 7 +++++++ arch/x86/include/asm/msr-index.h | 6 ++++++ 4 files changed, 32 insertions(+), 5 deletions(-)
arch/x86/events/core.c +3 -4
@@ -1236,8 +1236,7 @@ static inline void x86_assign_hw_event(struct perf_event *event,fallthrough ; case INTEL_PMC_IDX_FIXED ... INTEL_PMC_IDX_FIXED_BTS-1: hwc->config_base =MSR_ARCH_PERFMON_FIXED_CTR_CTRL ; - hwc->event_base =MSR_ARCH_PERFMON_FIXED_CTR0 + - (idx -INTEL_PMC_IDX_FIXED ); + hwc->event_base =x86_pmu_fixed_ctr_addr (idx -INTEL_PMC_IDX_FIXED ); hwc->event_base_rdpmc = (idx -INTEL_PMC_IDX_FIXED ) |INTEL_PMC_FIXED_RDPMC_BASE ;break ;
@@ -1573,7 +1572,7 @@ void perf_event_print_debug(void)for_each_set_bit (idx, fixed_cntr_mask,X86_PMC_IDX_MAX ) {if (fixed_counter_disabled (idx, cpuc->pmu))continue ; -rdmsrl (MSR_ARCH_PERFMON_FIXED_CTR0 + idx, pmc_count); +rdmsrl (x86_pmu_fixed_ctr_addr (idx), pmc_count);pr_info ("CPU#%d: fixed-PMC%d count: %016llx\n" , cpu, idx, pmc_count);
@@ -2483,7 +2482,7 @@ void perf_clear_dirty_counters(void)if (!test_bit (i -INTEL_PMC_IDX_FIXED ,hybrid (cpuc->pmu, fixed_cntr_mask)))continue ; -wrmsrl (MSR_ARCH_PERFMON_FIXED_CTR0 + (i -INTEL_PMC_IDX_FIXED ),0 ); +wrmsrl (x86_pmu_fixed_ctr_addr (i -INTEL_PMC_IDX_FIXED ),0 ); }else {wrmsrl (x86_pmu_event_addr (i),0 ); }
arch/x86/events/intel/core.c +16 -1
@@ -2953,7 +2953,7 @@ static void intel_pmu_reset(void)for_each_set_bit (idx, fixed_cntr_mask,INTEL_PMC_MAX_FIXED ) {if (fixed_counter_disabled (idx, cpuc->pmu))continue ; -wrmsrl_safe (MSR_ARCH_PERFMON_FIXED_CTR0 + idx,0ull ); +wrmsrl_safe (x86_pmu_fixed_ctr_addr (idx),0ull ); }if (ds)
@@ -5188,6 +5188,7 @@ static __initconst const struct x86_pmu core_pmu = { .schedule_events = x86_schedule_events, .eventsel = MSR_ARCH_PERFMON_EVENTSEL0, .perfctr = MSR_ARCH_PERFMON_PERFCTR0, + .fixedctr = MSR_ARCH_PERFMON_FIXED_CTR0, .event_map = intel_pmu_event_map, .max_events = ARRAY_SIZE(intel_perfmon_event_map), .apic = 1,
@@ -5241,6 +5242,7 @@ static __initconst const struct x86_pmu intel_pmu = { .schedule_events = x86_schedule_events, .eventsel = MSR_ARCH_PERFMON_EVENTSEL0, .perfctr = MSR_ARCH_PERFMON_PERFCTR0, + .fixedctr = MSR_ARCH_PERFMON_FIXED_CTR0, .event_map = intel_pmu_event_map, .max_events = ARRAY_SIZE(intel_perfmon_event_map), .apic = 1,
@@ -6176,6 +6178,11 @@ static void intel_pmu_check_extra_regs(struct extra_reg *extra_regs) } } +static inline int intel_pmu_addr_offset (int index,bool eventsel) +{ +return MSR_IA32_PMC_V6_STEP * index; +} +static const struct {enum hybrid_pmu_type id;char *name; } intel_hybrid_pmu_type_map[]__initconst = { { hybrid_small,"cpu_atom" }, { hybrid_big,"cpu_core" },
@@ -7150,6 +7157,14 @@ __init int intel_pmu_init(void)pr_cont ("full-width counters, " ); } +/* Support V6+ MSR Aliasing */ +if (x86_pmu.version >=6 ) { + x86_pmu.perfctr =MSR_IA32_PMC_V6_GP0_CTR ; + x86_pmu.eventsel =MSR_IA32_PMC_V6_GP0_CFG_A ; + x86_pmu.fixedctr =MSR_IA32_PMC_V6_FX0_CTR ; + x86_pmu.addr_offset = intel_pmu_addr_offset; + } +if (!is_hybrid () && x86_pmu.intel_cap.perf_metrics) x86_pmu.intel_ctrl |=1ULL <<GLOBAL_CTRL_EN_PERF_METRICS ;
arch/x86/events/perf_event.h +7 -0
@@ -787,6 +787,7 @@ struct x86_pmu {int (*schedule_events)(struct cpu_hw_events *cpuc,int n,int *assign);unsigned eventsel;unsigned perfctr; +unsigned fixedctr;int (*addr_offset)(int index,bool eventsel);int (*rdpmc_index)(int index);u64 (*event_map)(int);
@@ -1144,6 +1145,12 @@ static inline unsigned int x86_pmu_event_addr(int index) x86_pmu.addr_offset(index, false) : index); } +static inline unsigned int x86_pmu_fixed_ctr_addr (int index) +{ +return x86_pmu.fixedctr + (x86_pmu.addr_offset ? + x86_pmu.addr_offset (index, false) : index); +} +static inline int x86_pmu_rdpmc_index (int index) {return x86_pmu.rdpmc_index ? x86_pmu.rdpmc_index (index) : index;
arch/x86/include/asm/msr-index.h +6 -0
@@ -566,6 +566,12 @@#define MSR_RELOAD_PMC0 0x000014c1#define MSR_RELOAD_FIXED_CTR0 0x00001309 +/* V6 PMON MSR range */ +#define MSR_IA32_PMC_V6_GP0_CTR 0x1900 +#define MSR_IA32_PMC_V6_GP0_CFG_A 0x1901 +#define MSR_IA32_PMC_V6_FX0_CTR 0x1980 +#define MSR_IA32_PMC_V6_STEP 4 +/* KeyID partitioning between MKTME and TDX */ #define MSR_IA32_MKTME_KEYID_PARTITIONING 0x00000087
Signature · 1 line
2.35.1
PATCH v2 9/13 perf/x86: Extend event update interface
In reply to [email protected]From: Kan Liang <[email protected]>
The current event update interface directly reads the values from the counter, but the values may not be the accurate ones users require. For example, the sample read feature wants the counter value of the member events when the leader event is overflow. But with the current implementation, the read (event update) actually happens in the NMI handler. There may be a small gap between the overflow and the NMI handler. The new Intel PEBS counters snapshotting feature can provide the accurate counter value in the overflow. The event update interface has to be updated to apply the given accurate values.
Pass the accurate values via the event update interface. If the value is not available, still directly read the counter.
Using u64 * rather than u64 as the new parameter. Because 0 might be a valid rdpmc() value. The !val cannot be used to distinguish between there begin an argument and there not being one. Also, for some cases, e.g., intel_update_topdown_event, there could be more than one counter/register are read.
--- arch/x86/events/amd/core.c | 2 +- arch/x86/events/core.c | 13 ++++++----- arch/x86/events/intel/core.c | 40 +++++++++++++++++++--------------- arch/x86/events/intel/p4.c | 2 +- arch/x86/events/perf_event.h | 4 ++-- arch/x86/events/zhaoxin/core.c | 2 +- 6 files changed, 36 insertions(+), 27 deletions(-)
arch/x86/events/amd/core.c +1 -1
@@ -984,7 +984,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) event = cpuc->events[idx]; hwc = &event->hw; -x86_perf_event_update (event); +x86_perf_event_update (event,NULL ); mask =BIT_ULL (idx);if (!(status & mask))
arch/x86/events/core.c +8 -5
@@ -112,7 +112,7 @@ u64 __read_mostly hw_cache_extra_regs* Can only be executed on the CPU where the event is active. * Returns the delta events processed. */ -u64 x86_perf_event_update (struct perf_event *event) +u64 x86_perf_event_update (struct perf_event *event,u64 *val) {struct hw_perf_event *hwc = &event->hw;int shift =64 - x86_pmu.cntval_bits;
@@ -131,7 +131,10 @@ u64 x86_perf_event_update(struct perf_event *event)*/ prev_raw_count =local64_read (&hwc->prev_count);do { -rdpmcl (hwc->event_base_rdpmc, new_raw_count); +if (!val) +rdpmcl (hwc->event_base_rdpmc, new_raw_count); +else + new_raw_count = *val; }while (!local64_try_cmpxchg (&hwc->prev_count, &prev_raw_count, new_raw_count));
@@ -1598,7 +1601,7 @@ void x86_pmu_stop(struct perf_event *event, int flags)* Drain the remaining delta count out of a event * that we are disabling: */ -static_call (x86_pmu_update)(event); +static_call (x86_pmu_update)(event,NULL ); hwc->state |=PERF_HES_UPTODATE ; } }
@@ -1689,7 +1692,7 @@ int x86_pmu_handle_irq(struct pt_regs *regs) event = cpuc->events[idx]; - val =static_call (x86_pmu_update)(event); + val =static_call (x86_pmu_update)(event,NULL );if (val & (1ULL << (x86_pmu.cntval_bits -1 )))continue ;
@@ -2036,7 +2039,7 @@ static void x86_pmu_static_call_update(void)static void _x86_pmu_read (struct perf_event *event) { -static_call (x86_pmu_update)(event); +static_call (x86_pmu_update)(event,NULL ); }void x86_pmu_show_pmu_cap (struct pmu *pmu)
arch/x86/events/intel/core.c +23 -17
@@ -2418,7 +2418,7 @@ static void intel_pmu_nhm_workaround(void)for (i =0 ; i <4 ; i++) { event = cpuc->events[i];if (event) -static_call (x86_pmu_update)(event); +static_call (x86_pmu_update)(event,NULL ); }for (i =0 ; i <4 ; i++) {
@@ -2710,7 +2710,7 @@ static void update_saved_topdown_regs(struct perf_event *event, u64 slots,* modify by a NMI. PMU has to be disabled before calling this function. */ -static u64 intel_update_topdown_event (struct perf_event *event,int metric_end) +static u64 intel_update_topdown_event (struct perf_event *event,int metric_end,u64 *val) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);struct perf_event *other;
@@ -2718,13 +2718,18 @@ static u64 intel_update_topdown_event(struct perf_event *event, int metric_end)bool reset = true;int idx; -/* read Fixed counter 3 */ -rdpmcl ((3 |INTEL_PMC_FIXED_RDPMC_BASE ), slots); -if (!slots) -return 0 ; +if (!val) { +/* read Fixed counter 3 */ +rdpmcl ((3 |INTEL_PMC_FIXED_RDPMC_BASE ), slots); +if (!slots) +return 0 ; -/* read PERF_METRICS */ -rdpmcl (INTEL_PMC_FIXED_RDPMC_METRICS , metrics); +/* read PERF_METRICS */ +rdpmcl (INTEL_PMC_FIXED_RDPMC_METRICS , metrics); + }else { + slots = val[0 ]; + metrics = val[1 ]; + }for_each_set_bit (idx, cpuc->active_mask, metric_end +1 ) {if (!is_topdown_idx (idx))
@@ -2767,10 +2772,11 @@ static u64 intel_update_topdown_event(struct perf_event *event, int metric_end)return slots; } -static u64 icl_update_topdown_event (struct perf_event *event) +static u64 icl_update_topdown_event (struct perf_event *event,u64 *val) {return intel_update_topdown_event (event,INTEL_PMC_IDX_METRIC_BASE + - x86_pmu.num_topdown_events -1 ); + x86_pmu.num_topdown_events -1 , + val); }DEFINE_STATIC_CALL (intel_pmu_update_topdown_event, x86_perf_event_update);
@@ -2785,7 +2791,7 @@ static void intel_pmu_read_topdown_event(struct perf_event *event)return ;perf_pmu_disable (event->pmu); -static_call (intel_pmu_update_topdown_event)(event); +static_call (intel_pmu_update_topdown_event)(event,NULL );perf_pmu_enable (event->pmu); }
@@ -2796,7 +2802,7 @@ static void intel_pmu_read_event(struct perf_event *event)else if (is_topdown_count (event))intel_pmu_read_topdown_event (event);else -x86_perf_event_update (event); +x86_perf_event_update (event,NULL ); }static void intel_pmu_enable_fixed (struct perf_event *event)
@@ -2899,7 +2905,7 @@ static void intel_pmu_add_event(struct perf_event *event)*/ int intel_pmu_save_and_restart (struct perf_event *event) { -static_call (x86_pmu_update)(event); +static_call (x86_pmu_update)(event,NULL );/* * For a checkpointed counter always reset back to 0. This * avoids a situation where the counter overflows, aborts the
@@ -2922,12 +2928,12 @@ static int intel_pmu_set_period(struct perf_event *event)return x86_perf_event_set_period (event); } -static u64 intel_pmu_update (struct perf_event *event) +static u64 intel_pmu_update (struct perf_event *event,u64 *val) {if (unlikely (is_topdown_count (event))) -return static_call (intel_pmu_update_topdown_event)(event); +return static_call (intel_pmu_update_topdown_event)(event, val); -return x86_perf_event_update (event); +return x86_perf_event_update (event, val); }static void intel_pmu_reset (void )
@@ -3091,7 +3097,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)*/ if (__test_and_clear_bit (GLOBAL_STATUS_PERF_METRICS_OVF_BIT , (unsigned long *)&status)) { handled++; -static_call (intel_pmu_update_topdown_event)(NULL ); +static_call (intel_pmu_update_topdown_event)(NULL ,NULL ); }/*
arch/x86/events/intel/p4.c +1 -1
@@ -1058,7 +1058,7 @@ static int p4_pmu_handle_irq(struct pt_regs *regs)/* it might be unflagged overflow */ overflow =p4_pmu_clear_cccr_ovf (hwc); - val =x86_perf_event_update (event); + val =x86_perf_event_update (event,NULL );if (!overflow && (val & (1ULL << (x86_pmu.cntval_bits -1 ))))continue ;
arch/x86/events/perf_event.h +2 -2
@@ -782,7 +782,7 @@ struct x86_pmu {void (*del)(struct perf_event *);void (*read)(struct perf_event *event);int (*set_period)(struct perf_event *event); -u64 (*update)(struct perf_event *event); +u64 (*update)(struct perf_event *event,u64 *val);int (*hw_config)(struct perf_event *event);int (*schedule_events)(struct cpu_hw_events *cpuc,int n,int *assign);unsigned eventsel;
@@ -1131,7 +1131,7 @@ extern u64 __read_mostly hw_cache_extra_regs [PERF_COUNT_HW_CACHE_OP_MAX ] [PERF_COUNT_HW_CACHE_RESULT_MAX ]; -u64 x86_perf_event_update (struct perf_event *event); +u64 x86_perf_event_update (struct perf_event *event,u64 *cntr);static inline unsigned int x86_pmu_config_addr (int index) {
arch/x86/events/zhaoxin/core.c +1 -1
@@ -391,7 +391,7 @@ static int zhaoxin_pmu_handle_irq(struct pt_regs *regs)if (!test_bit (bit, cpuc->active_mask))continue ; -x86_perf_event_update (event); +x86_perf_event_update (event,NULL );perf_sample_data_init (&data,0 , event->hw.last_period);if (!x86_perf_event_set_period (event))
Signature · 1 line
2.35.1
PATCH v2 10/13 perf: Extend perf_output_read
In reply to [email protected]From: Kan Liang <[email protected]>
The event may have been updated in the PMU-specific implementation, e.g., Intel PEBS counters snapshotting. The common code should not read and overwrite the value.
The PERF_SAMPLE_READ in the data->sample_type can be used to detect whether the PMU-specific value is available. If yes, avoid the pmu->read() in the common code.
--- kernel/events/core.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
kernel/events/core.c +8 -7
@@ -7243,7 +7243,7 @@ static void perf_output_read_one(struct perf_output_handle *handle,static void perf_output_read_group (struct perf_output_handle *handle,struct perf_event *event, -u64 enabled,u64 running) +u64 enabled,u64 running,bool read) {struct perf_event *leader = event->group_leader, *sub;u64 read_format = event->attr.read_format;
@@ -7265,7 +7265,7 @@ static void perf_output_read_group(struct perf_output_handle *handle,if (read_format &PERF_FORMAT_TOTAL_TIME_RUNNING ) values[n++] = running; -if ((leader != event) && +if ((leader != event) && read && (leader->state ==PERF_EVENT_STATE_ACTIVE )) leader->pmu->read (leader);
@@ -7280,7 +7280,7 @@ static void perf_output_read_group(struct perf_output_handle *handle,for_each_sibling_event (sub, leader) { n =0 ; -if ((sub != event) && +if ((sub != event) && read && (sub->state ==PERF_EVENT_STATE_ACTIVE )) sub->pmu->read (sub);
@@ -7307,7 +7307,8 @@ static void perf_output_read_group(struct perf_output_handle *handle,* on another CPU, from interrupt/NMI context. */ static void perf_output_read (struct perf_output_handle *handle, -struct perf_event *event) +struct perf_event *event, +bool read) {u64 enabled =0 , running =0 , now;u64 read_format = event->attr.read_format;
@@ -7325,7 +7326,7 @@ static void perf_output_read(struct perf_output_handle *handle,calc_timer_values (event, &now, &enabled, &running);if (event->attr.read_format &PERF_FORMAT_GROUP ) -perf_output_read_group (handle, event, enabled, running); +perf_output_read_group (handle, event, enabled, running, read);else perf_output_read_one (handle, event, enabled, running); }
@@ -7367,7 +7368,7 @@ void perf_output_sample(struct perf_output_handle *handle,perf_output_put (handle, data->period);if (sample_type &PERF_SAMPLE_READ ) -perf_output_read (handle, event); +perf_output_read (handle, event, !(data->sample_flags &PERF_SAMPLE_READ ));if (sample_type &PERF_SAMPLE_CALLCHAIN ) {int size =1 ;
@@ -7968,7 +7969,7 @@ perf_event_read_event(struct perf_event *event,return ;perf_output_put (&handle, read_event); -perf_output_read (&handle, event); +perf_output_read (&handle, event, true);perf_event__output_id_sample (event, &handle, &sample);perf_output_end (&handle);
Signature · 1 line
2.35.1
PATCH v2 2/13 perf/x86: Support counter mask
In reply to [email protected]From: Kan Liang <[email protected]>
The current perf assumes that both GP and fixed counters are contiguous. But it's not guaranteed on newer Intel platforms or in a virtualization environment.
Use the counter mask to replace the number of counters for both GP and the fixed counters. For the other ARCHs or old platforms which don't support a counter mask, using GENMASK_ULL(num_counter - 1, 0) to replace. There is no functional change for them.
The interface to KVM is not changed. The number of counters still be passed to KVM. It can be updated later separately.
--- arch/x86/events/amd/core.c | 24 ++--- arch/x86/events/core.c | 98 ++++++++++---------- arch/x86/events/intel/core.c | 164 ++++++++++++++++----------------- arch/x86/events/intel/ds.c | 19 ++-- arch/x86/events/intel/knc.c | 2 +- arch/x86/events/intel/p4.c | 10 +- arch/x86/events/intel/p6.c | 2 +- arch/x86/events/perf_event.h | 47 ++++++++-- arch/x86/events/zhaoxin/core.c | 12 +-- 9 files changed, 199 insertions(+), 179 deletions(-)
arch/x86/events/amd/core.c +12 -12
@@ -432,7 +432,7 @@ static void __amd_put_nb_event_constraints(struct cpu_hw_events *cpuc,* be removed on one CPU at a time AND PMU is disabled * when we come here */ -for (i =0 ; i < x86_pmu.num_counters; i++) { +for_each_set_bit (i, x86_pmu.cntr_mask,X86_PMC_IDX_MAX ) {if (cmpxchg (nb->owners + i, event,NULL ) == event)break ; }
@@ -499,7 +499,7 @@ __amd_get_nb_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *ev* because of successive calls to x86_schedule_events() from * hw_perf_group_sched_in() without hw_perf_enable() */ -for_each_set_bit (idx, c->idxmsk, x86_pmu.num_counters) { +for_each_set_bit (idx, c->idxmsk,x86_pmu_max_num_counters (NULL )) {if (new ==-1 || hwc->idx == idx)/* assign free slot, prefer hwc->idx */ old =cmpxchg (nb->owners + idx,NULL , event);
@@ -542,7 +542,7 @@ static struct amd_nb *amd_alloc_nb(int cpu)/* * initialize all possible NB constraints */ -for (i =0 ; i < x86_pmu.num_counters; i++) { +for_each_set_bit (i, x86_pmu.cntr_mask,X86_PMC_IDX_MAX ) {__set_bit (i, nb->event_constraints[i].idxmsk); nb->event_constraints[i].weight =1 ; }
@@ -735,7 +735,7 @@ static void amd_pmu_check_overflow(void)* counters are always enabled when this function is called and * ARCH_PERFMON_EVENTSEL_INT is always set. */ -for (idx =0 ; idx < x86_pmu.num_counters; idx++) { +for_each_set_bit (idx, x86_pmu.cntr_mask,X86_PMC_IDX_MAX ) {if (!test_bit (idx, cpuc->active_mask))continue ;
@@ -755,7 +755,7 @@ static void amd_pmu_enable_all(int added)amd_brs_enable_all (); -for (idx =0 ; idx < x86_pmu.num_counters; idx++) { +for_each_set_bit (idx, x86_pmu.cntr_mask,X86_PMC_IDX_MAX ) {/* only activate events which are marked as active */ if (!test_bit (idx, cpuc->active_mask))continue ;
@@ -978,7 +978,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs)/* Clear any reserved bits set by buggy microcode */ status &= amd_pmu_global_cntr_mask; -for (idx =0 ; idx < x86_pmu.num_counters; idx++) { +for_each_set_bit (idx, x86_pmu.cntr_mask,X86_PMC_IDX_MAX ) {if (!test_bit (idx, cpuc->active_mask))continue ;
@@ -1313,7 +1313,7 @@ static __initconst const struct x86_pmu amd_pmu = { .addr_offset = amd_pmu_addr_offset, .event_map = amd_pmu_event_map, .max_events = ARRAY_SIZE(amd_perfmon_event_map), - .num_counters = AMD64_NUM_COUNTERS, + .cntr_mask64 = GENMASK_ULL(AMD64_NUM_COUNTERS - 1, 0), .add = amd_pmu_add_event, .del = amd_pmu_del_event, .cntval_bits = 48,
@@ -1412,7 +1412,7 @@ static int __init amd_core_pmu_init(void)*/ x86_pmu.eventsel =MSR_F15H_PERF_CTL ; x86_pmu.perfctr =MSR_F15H_PERF_CTR ; - x86_pmu.num_counters =AMD64_NUM_COUNTERS_CORE ; + x86_pmu.cntr_mask64 =GENMASK_ULL (AMD64_NUM_COUNTERS_CORE -1 ,0 );/* Check for Performance Monitoring v2 support */ if (boot_cpu_has (X86_FEATURE_PERFMON_V2 )) {
@@ -1422,9 +1422,9 @@ static int __init amd_core_pmu_init(void) x86_pmu.version =2 ;/* Find the number of available Core PMCs */ - x86_pmu.num_counters = ebx.split.num_core_pmc; + x86_pmu.cntr_mask64 =GENMASK_ULL (ebx.split.num_core_pmc -1 ,0 ); - amd_pmu_global_cntr_mask = (1ULL << x86_pmu.num_counters) -1 ; + amd_pmu_global_cntr_mask = x86_pmu.cntr_mask64;/* Update PMC handling functions */ x86_pmu.enable_all = amd_pmu_v2_enable_all;
@@ -1452,12 +1452,12 @@ static int __init amd_core_pmu_init(void)* even numbered counter that has a consecutive adjacent odd * numbered counter following it. */ -for (i =0 ; i < x86_pmu.num_counters -1 ; i +=2 ) +for (i =0 ; i <x86_pmu_max_num_counters (NULL ) -1 ; i +=2 ) even_ctr_mask |=BIT_ULL (i); pair_constraint = (struct event_constraint )__EVENT_CONSTRAINT (0 , even_ctr_mask,0 , - x86_pmu.num_counters /2 ,0 , +x86_pmu_max_num_counters (NULL ) /2 ,0 ,PERF_X86_EVENT_PAIR ); x86_pmu.get_event_constraints = amd_get_event_constraints_f17h;
arch/x86/events/core.c +47 -51
270 lines not on this page · raw message
arch/x86/events/intel/core.c +80 -84
321 lines not on this page · raw message
arch/x86/events/intel/ds.c +8 -11
@@ -1138,7 +1138,6 @@ static inline void pebs_update_threshold(struct cpu_hw_events *cpuc) {struct debug_store *ds = cpuc->ds;int max_pebs_events =intel_pmu_max_num_pebs (cpuc->pmu); -int num_counters_fixed =hybrid (cpuc->pmu, num_counters_fixed);u64 threshold;int reserved;
@@ -1146,7 +1145,7 @@ static inline void pebs_update_threshold(struct cpu_hw_events *cpuc)return ;if (x86_pmu.flags &PMU_FL_PEBS_ALL ) - reserved = max_pebs_events + num_counters_fixed; + reserved = max_pebs_events +x86_pmu_max_num_counters_fixed (cpuc->pmu);else reserved = max_pebs_events;
@@ -2172,8 +2171,8 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs, struct perf_sample_d mask = x86_pmu.pebs_events_mask; size = max_pebs_events;if (x86_pmu.flags &PMU_FL_PEBS_ALL ) { - mask |= ((1ULL << x86_pmu.num_counters_fixed) -1 ) <<INTEL_PMC_IDX_FIXED ; - size =INTEL_PMC_IDX_FIXED + x86_pmu.num_counters_fixed; + mask |= x86_pmu.fixed_cntr_mask64 <<INTEL_PMC_IDX_FIXED ; + size =INTEL_PMC_IDX_FIXED +x86_pmu_max_num_counters_fixed (NULL ); }if (unlikely (base >= top)) {
@@ -2269,11 +2268,10 @@ static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_d {short counts[INTEL_PMC_IDX_FIXED +MAX_FIXED_PEBS_EVENTS ] = {};struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); -int num_counters_fixed =hybrid (cpuc->pmu, num_counters_fixed);struct debug_store *ds = cpuc->ds;struct perf_event *event;void *base, *at, *top; -int bit, size; +int bit;u64 mask;if (!x86_pmu.pebs_active)
@@ -2285,11 +2283,10 @@ static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_d ds->pebs_index = ds->pebs_buffer_base; mask =hybrid (cpuc->pmu, pebs_events_mask) | - (((1ULL << num_counters_fixed) -1 ) <<INTEL_PMC_IDX_FIXED ); - size =INTEL_PMC_IDX_FIXED + num_counters_fixed; + (hybrid (cpuc->pmu, fixed_cntr_mask64) <<INTEL_PMC_IDX_FIXED );if (unlikely (base >= top)) { -intel_pmu_pebs_event_update_no_drain (cpuc, size); +intel_pmu_pebs_event_update_no_drain (cpuc,X86_PMC_IDX_MAX );return ; }
@@ -2299,11 +2296,11 @@ static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_d pebs_status =get_pebs_status (at) & cpuc->pebs_enabled; pebs_status &= mask; -for_each_set_bit (bit, (unsigned long *)&pebs_status, size) +for_each_set_bit (bit, (unsigned long *)&pebs_status,X86_PMC_IDX_MAX ) counts[bit]++; } -for_each_set_bit (bit, (unsigned long *)&mask, size) { +for_each_set_bit (bit, (unsigned long *)&mask,X86_PMC_IDX_MAX ) {if (counts[bit] ==0 )continue ;
arch/x86/events/intel/knc.c +1 -1
@@ -303,7 +303,7 @@ static const struct x86_pmu knc_pmu __initconst = { .apic = 1, .max_period = (1ULL << 39) - 1, .version = 0, - .num_counters = 2, + .cntr_mask64 = 0x3, .cntval_bits = 40, .cntval_mask = (1ULL << 40) - 1, .get_event_constraints = x86_get_event_constraints,
arch/x86/events/intel/p4.c +5 -5
@@ -919,7 +919,7 @@ static void p4_pmu_disable_all(void)struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);int idx; -for (idx =0 ; idx < x86_pmu.num_counters; idx++) { +for_each_set_bit (idx, x86_pmu.cntr_mask,X86_PMC_IDX_MAX ) {struct perf_event *event = cpuc->events[idx];if (!test_bit (idx, cpuc->active_mask))continue ;
@@ -998,7 +998,7 @@ static void p4_pmu_enable_all(int added)struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);int idx; -for (idx =0 ; idx < x86_pmu.num_counters; idx++) { +for_each_set_bit (idx, x86_pmu.cntr_mask,X86_PMC_IDX_MAX ) {struct perf_event *event = cpuc->events[idx];if (!test_bit (idx, cpuc->active_mask))continue ;
@@ -1040,7 +1040,7 @@ static int p4_pmu_handle_irq(struct pt_regs *regs) cpuc =this_cpu_ptr (&cpu_hw_events); -for (idx =0 ; idx < x86_pmu.num_counters; idx++) { +for_each_set_bit (idx, x86_pmu.cntr_mask,X86_PMC_IDX_MAX ) {int overflow;if (!test_bit (idx, cpuc->active_mask)) {
@@ -1353,7 +1353,7 @@ static __initconst const struct x86_pmu p4_pmu = {* though leave it restricted at moment assuming * HT is on */ - .num_counters = ARCH_P4_MAX_CCCR, + .cntr_mask64 = GENMASK_ULL(ARCH_P4_MAX_CCCR - 1, 0), .apic = 1, .cntval_bits = ARCH_P4_CNTRVAL_BITS, .cntval_mask = ARCH_P4_CNTRVAL_MASK,
@@ -1395,7 +1395,7 @@ __init int p4_pmu_init(void)* * Solve this by zero'ing out the registers to mimic a reset. */ -for (i =0 ; i < x86_pmu.num_counters; i++) { +for_each_set_bit (i, x86_pmu.cntr_mask,X86_PMC_IDX_MAX ) { reg =x86_pmu_config_addr (i);wrmsrl_safe (reg,0ULL ); }
arch/x86/events/intel/p6.c +1 -1
@@ -214,7 +214,7 @@ static __initconst const struct x86_pmu p6_pmu = { .apic = 1, .max_period = (1ULL << 31) - 1, .version = 0, - .num_counters = 2, + .cntr_mask64 = 0x3,/* * Events have 40 bits implemented. However they are designed such * that bits [32-39] are sign extensions of bit 31. As such the
arch/x86/events/perf_event.h +39 -8
71 lines not on this page · raw message
arch/x86/events/zhaoxin/core.c +6 -6
@@ -530,13 +530,13 @@ __init int zhaoxin_pmu_init(void)pr_info ("Version check pass!\n" ); x86_pmu.version = version; - x86_pmu.num_counters = eax.split.num_counters; + x86_pmu.cntr_mask64 =GENMASK_ULL (eax.split.num_counters -1 ,0 ); x86_pmu.cntval_bits = eax.split.bit_width; x86_pmu.cntval_mask = (1ULL << eax.split.bit_width) -1 ; x86_pmu.events_maskl = ebx.full; x86_pmu.events_mask_len = eax.split.mask_length; - x86_pmu.num_counters_fixed = edx.split.num_counters_fixed; + x86_pmu.fixed_cntr_mask64 =GENMASK_ULL (edx.split.num_counters_fixed -1 ,0 );x86_add_quirk (zhaoxin_arch_events_quirk);switch (boot_cpu_data.x86) {
@@ -604,13 +604,13 @@ __init int zhaoxin_pmu_init(void)return -ENODEV ; } - x86_pmu.intel_ctrl = (1 << (x86_pmu.num_counters)) -1 ; - x86_pmu.intel_ctrl |= ((1LL << x86_pmu.num_counters_fixed)-1 ) <<INTEL_PMC_IDX_FIXED ; + x86_pmu.intel_ctrl = x86_pmu.cntr_mask64; + x86_pmu.intel_ctrl |= x86_pmu.fixed_cntr_mask64 <<INTEL_PMC_IDX_FIXED ;if (x86_pmu.event_constraints) {for_each_event_constraint (c, x86_pmu.event_constraints) { - c->idxmsk64 |= (1ULL << x86_pmu.num_counters) -1 ; - c->weight += x86_pmu.num_counters; + c->idxmsk64 |= x86_pmu.cntr_mask64; + c->weight +=x86_pmu_num_counters (NULL ); } }
Signature · 1 line
2.35.1
PATCH v2 11/13 perf/x86/intel: Move PEBS event update after the sample output
In reply to [email protected]From: Kan Liang <[email protected]>
In the drain_pebs(), besides outputting the sample data, the perf needs to update the PEBS event (e.g., prev_count, event->count, etc.) as well. Both operations may invoke the perf_event_update(), but the sequence of the two operations doesn't matter for now. Because the updated event value is read directly from the counter via rdpmc. The counter stops in the drain_pebs().
But if the updated event value is from different places (PEBS record VS. counter), the sequence does matter. For example, with the new Intel PEBS counters snapshotting feature, the large PEBS can be enabled for the sample read, since counter values for each sample are recorded in PEBS records. The current perf does the PEBS event update first, which also updates the event for all the records altogether. It's impossible for the later sample read output to dump the value for each sample, since the prev_count is already the newest one from the current counter.
Move PEBS event update after the sample output. For each sample read output, it will update and output the value only for this sample (according to the value in the PEBS record). Once all samples are output, update the PEBS event again according to the current counter, and set the left period.
The !intel_pmu_save_and_restart() only happens when !hwc->event_base or the left > 0. The !hwc->event_base is impossible for the PEBS event which is only available on GP and fixed counters.
The __intel_pmu_pebs_event() is only to process the overflowed sample. The left should be always <=0.
It's safe to ignore the return from the !inel_pmu_save_and_restart() check.
--- arch/x86/events/intel/ds.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-)
arch/x86/events/intel/ds.c +11 -11
@@ -2158,17 +2158,6 @@ __intel_pmu_pebs_event(struct perf_event *event,void *at =get_next_pebs_record_by_bit (base, top, bit);static struct pt_regs dummy_iregs; -if (hwc->flags &PERF_X86_EVENT_AUTO_RELOAD ) { -/* -* Now, auto-reload is only enabled in fixed period mode. -* The reload value is always hwc->sample_period. -* May need to change it, if auto-reload is enabled in -* freq mode later. -*/ -intel_pmu_save_and_restart_reload (event, count); - }else if (!intel_pmu_save_and_restart (event)) -return ; -if (!iregs) iregs = &dummy_iregs;
@@ -2197,6 +2186,17 @@ __intel_pmu_pebs_event(struct perf_event *event,if (perf_event_overflow (event, data, regs))x86_pmu_stop (event,0 ); } + +if (hwc->flags &PERF_X86_EVENT_AUTO_RELOAD ) { +/* +* Now, auto-reload is only enabled in fixed period mode. +* The reload value is always hwc->sample_period. +* May need to change it, if auto-reload is enabled in +* freq mode later. +*/ +intel_pmu_save_and_restart_reload (event, count); + }else +intel_pmu_save_and_restart (event); }static void intel_pmu_drain_pebs_core (struct pt_regs *iregs,struct perf_sample_data *data)
Signature · 1 line
2.35.1
PATCH v2 12/13 perf/x86/intel: Support PEBS counters snapshotting
In reply to [email protected]From: Kan Liang <[email protected]>
The counters snapshotting is a new adaptive PEBS extension, which can capture programmable counters, fixed-function counters, and performance metrics in a PEBS record. The feature is available in the PEBS format V6.
The target counters can be configured in the new fields of MSR_PEBS_CFG. Then the PEBS HW will generate the bit mask of counters (Counters Group Header) followed by the content of all the requested counters into a PEBS record.
The current Linux perf sample read feature intends to read the counters of other member events when the leader event is overflowing. But the current read is in the NMI handler, which may has a small gap from overflow. Using the counters snapshotting feature for the sample read.
Add a new PEBS_CNTR flag to indicate a sample read group that utilizes the counters snapshotting feature. When the group is scheduled, the PEBS configure can be updated accordingly.
--- arch/x86/events/intel/core.c | 33 ++++++++- arch/x86/events/intel/ds.c | 114 +++++++++++++++++++++++++++-- arch/x86/events/perf_event.h | 3 + arch/x86/events/perf_event_flags.h | 2 +- arch/x86/include/asm/perf_event.h | 15 ++++ 5 files changed, 157 insertions(+), 10 deletions(-)
arch/x86/events/intel/core.c +32 -1
@@ -4058,6 +4058,19 @@ static int intel_pmu_hw_config(struct perf_event *event) event->hw.flags |=PERF_X86_EVENT_PEBS_VIA_PT ; } +if ((event->attr.sample_type &PERF_SAMPLE_READ ) && + (x86_pmu.intel_cap.pebs_format >=6 )) { +struct perf_event *leader = event->group_leader; + +if (is_slots_event (leader)) + leader =list_next_entry (leader, sibling_list); + +if (leader->attr.precise_ip) { + leader->hw.flags |=PERF_X86_EVENT_PEBS_CNTR ; + event->hw.flags |=PERF_X86_EVENT_PEBS_CNTR ; + } + } +if ((event->attr.type ==PERF_TYPE_HARDWARE ) || (event->attr.type ==PERF_TYPE_HW_CACHE ))return 0 ;
@@ -4161,6 +4174,24 @@ static int intel_pmu_hw_config(struct perf_event *event)return 0 ; } +static int intel_pmu_schedule_events (struct cpu_hw_events *cpuc,int n,int *assign) +{ +struct perf_event *event; +int ret =x86_schedule_events (cpuc, n, assign); + +if (ret) +return ret; + +if (cpuc->is_fake) +return ret; + + event = cpuc->event_list[n -1 ]; +if (event && (event->hw.flags &PERF_X86_EVENT_PEBS_CNTR )) +intel_pmu_pebs_update_cfg (cpuc, n, assign); + +return 0 ; +} +/* * Currently, the only caller of this function is the atomic_switch_perf_msrs(). * The host perf context helps to prepare the values of the real hardware for
@@ -5245,7 +5276,7 @@ static __initconst const struct x86_pmu intel_pmu = { .set_period = intel_pmu_set_period, .update = intel_pmu_update, .hw_config = intel_pmu_hw_config, - .schedule_events = x86_schedule_events, + .schedule_events = intel_pmu_schedule_events, .eventsel = MSR_ARCH_PERFMON_EVENTSEL0, .perfctr = MSR_ARCH_PERFMON_PERFCTR0, .fixedctr = MSR_ARCH_PERFMON_FIXED_CTR0,
arch/x86/events/intel/ds.c +106 -8
148 lines not on this page · raw message
arch/x86/events/perf_event.h +3 -0
@@ -1132,6 +1132,7 @@ extern u64 __read_mostly hw_cache_extra_regs [PERF_COUNT_HW_CACHE_RESULT_MAX ];u64 x86_perf_event_update (struct perf_event *event,u64 *cntr); +DECLARE_STATIC_CALL (intel_pmu_update_topdown_event, x86_perf_event_update);static inline unsigned int x86_pmu_config_addr (int index) {
@@ -1626,6 +1627,8 @@ void intel_pmu_pebs_disable_all(void);void intel_pmu_pebs_sched_task (struct perf_event_pmu_context *pmu_ctx,bool sched_in); +void intel_pmu_pebs_update_cfg (struct cpu_hw_events *cpuc,int n,int *assign); +void intel_pmu_auto_reload_read (struct perf_event *event);void intel_pmu_store_pebs_lbrs (struct lbr_entry *lbr);
arch/x86/events/perf_event_flags.h +1 -1
@@ -9,7 +9,7 @@ PERF_ARCH(PEBS_LD_HSW, 0x00008) /* haswell style datala, load */ PERF_ARCH(PEBS_NA_HSW, 0x00010) /* haswell style datala, unknown */ PERF_ARCH(EXCL, 0x00020) /* HT exclusivity on counter */ PERF_ARCH(DYNAMIC, 0x00040) /* dynamic alloc'd constraint */ - /* 0x00080 */ +PERF_ARCH(PEBS_CNTR, 0x00080) /* PEBS counters snapshot */ PERF_ARCH(EXCL_ACCT, 0x00100) /* accounted EXCL event */ PERF_ARCH(AUTO_RELOAD, 0x00200) /* use PEBS auto-reload */PERF_ARCH (LARGE_PEBS ,0x00400 )/* use large PEBS */
arch/x86/include/asm/perf_event.h +15 -0
@@ -140,6 +140,12 @@#define PEBS_DATACFG_XMMS BIT_ULL(2)#define PEBS_DATACFG_LBRS BIT_ULL(3)#define PEBS_DATACFG_LBR_SHIFT 24 +#define PEBS_DATACFG_CNTR BIT_ULL(4) +#define PEBS_DATACFG_CNTR_SHIFT 32 +#define PEBS_DATACFG_CNTR_MASK GENMASK_ULL(15, 0) +#define PEBS_DATACFG_FIX_SHIFT 48 +#define PEBS_DATACFG_FIX_MASK GENMASK_ULL(7, 0) +#define PEBS_DATACFG_METRICS BIT_ULL(5)/* Steal the highest bit of pebs_data_cfg for SW usage */ #define PEBS_UPDATE_DS_SW BIT_ULL(63)
@@ -444,6 +450,15 @@ struct pebs_xmm {u64 xmm[16 *2 ];/* two entries for each register */ }; +struct pebs_cntr_header { +u32 cntr; +u32 fixed; +u32 metrics; +u32 reserved; +}; + +#define INTEL_CNTR_METRICS 0x3 +/* * AMD Extended Performance Monitoring and Debug cpuid feature detection */
Signature · 1 line
2.35.1
PATCH v2 13/13 perf/x86/intel: Support RDPMC metrics clear mode
In reply to [email protected]From: Kan Liang <[email protected]>
The new RDPMC enhancement, metrics clear mode, is to clear the PERF_METRICS-related resources as well as the fixed-function performance monitoring counter 3 after the read is performed. It is available for ring 3. The feature is enumerated by the IA32_PERF_CAPABILITIES.RDPMC_CLEAR_METRICS[bit 19]. To enable the feature, the IA32_FIXED_CTR_CTRL.METRICS_CLEAR_EN[bit 14] must be set.
Two ways were considered to enable the feature.
- Expose a knob in the sysfs globally. One user may affect the measurement of other users when changing the knob. The solution is dropped.
- Introduce a new event format, metrics_clear, for the slots event to disable/enable the feature only for the current process. Users can utilize the feature as needed.
The latter solution is implemented in the patch.
The current KVM doesn't support the perf metrics yet. For virtualization, the feature can be enabled later separately.
Update the document of perf metrics.
--- arch/x86/events/intel/core.c | 20 +++++++++++++++++++- arch/x86/events/perf_event.h | 1 + arch/x86/include/asm/perf_event.h | 4 ++++ tools/perf/Documentation/topdown.txt | 9 +++++++-- 4 files changed, 31 insertions(+), 3 deletions(-)
arch/x86/events/intel/core.c +19 -1
@@ -2822,6 +2822,9 @@ static void intel_pmu_enable_fixed(struct perf_event *event)return ; idx =INTEL_PMC_IDX_FIXED_SLOTS ; + +if (event->attr.config1 &INTEL_TD_CFG_METRIC_CLEAR ) + bits |=INTEL_FIXED_3_METRICS_CLEAR ; }intel_set_masks (event, idx);
@@ -4086,7 +4089,12 @@ static int intel_pmu_hw_config(struct perf_event *event)* is used in a metrics group, it too cannot support sampling. */ if (intel_pmu_has_cap (event,PERF_CAP_METRICS_IDX ) &&is_topdown_event (event)) { -if (event->attr.config1 || event->attr.config2) +/* The metrics_clear can only be set for the slots event */ +if (event->attr.config1 && + (!is_slots_event (event) || (event->attr.config1 & ~INTEL_TD_CFG_METRIC_CLEAR ))) +return -EINVAL ; + +if (event->attr.config2)return -EINVAL ;/*
@@ -4673,6 +4681,8 @@ PMU_FORMAT_ATTR(in_tx, "config:32" );PMU_FORMAT_ATTR (in_tx_cp,"config:33" );PMU_FORMAT_ATTR (eq,"config:36" );/* v6 + */ +PMU_FORMAT_ATTR (metrics_clear,"config1:0" );/* PERF_CAPABILITIES.RDPMC_METRICS_CLEAR */ +static ssize_t umask2_show (struct device *dev,struct device_attribute *attr,char *page)
@@ -4692,6 +4702,7 @@ static struct device_attribute format_attr_umask2 =static struct attribute *format_evtsel_ext_attrs[] = { &format_attr_umask2.attr, &format_attr_eq.attr, + &format_attr_metrics_clear.attr,NULL };
@@ -4716,6 +4727,13 @@ evtsel_ext_is_visible(struct kobject *kobj, struct attribute *attr, int i)if (i ==1 )return (mask &ARCH_PERFMON_EVENTSEL_EQ ) ? attr->mode :0 ; +/* PERF_CAPABILITIES.RDPMC_METRICS_CLEAR */ +if (i ==2 ) { +union perf_capabilities intel_cap =hybrid (dev_get_drvdata (dev), intel_cap); + +return intel_cap.rdpmc_metrics_clear ? attr->mode :0 ; + } +return 0 ; }
arch/x86/events/perf_event.h +1 -0
@@ -624,6 +624,7 @@ union perf_capabilities { u64 pebs_output_pt_available:1; u64 pebs_timing_info:1; u64 anythread_deprecated:1; + u64 rdpmc_metrics_clear:1; };u64 capabilities; };
arch/x86/include/asm/perf_event.h +4 -0
@@ -41,6 +41,7 @@#define INTEL_FIXED_0_USER (1ULL << 1)#define INTEL_FIXED_0_ANYTHREAD (1ULL << 2)#define INTEL_FIXED_0_ENABLE_PMI (1ULL << 3) +#define INTEL_FIXED_3_METRICS_CLEAR (1ULL << 2)#define HSW_IN_TX (1ULL << 32)#define HSW_IN_TX_CHECKPOINTED (1ULL << 33)
@@ -378,6 +379,9 @@ static inline bool use_fixed_pseudo_encoding(u64 code)#define INTEL_TD_METRIC_MAX INTEL_TD_METRIC_MEM_BOUND#define INTEL_TD_METRIC_NUM 8 +#define INTEL_TD_CFG_METRIC_CLEAR_BIT 0 +#define INTEL_TD_CFG_METRIC_CLEAR BIT_ULL(INTEL_TD_CFG_METRIC_CLEAR_BIT) +static inline bool is_metric_idx (int idx) {return (unsigned )(idx -INTEL_PMC_IDX_METRIC_BASE ) <INTEL_TD_METRIC_NUM ;
tools/perf/Documentation/topdown.txt +7 -2
@@ -280,8 +280,13 @@ with no longer interval than a few seconds perf stat -I 1000 --topdown ... -For user programs using RDPMC directly the counter can -be reset explicitly using ioctl: +Starting from the Lunar Lake p-core, a RDPMC metrics clear mode is +introduced. The metrics and the fixed counter 3 are automatically +cleared after the read is performed. It is recommended to always enable +the mode. To enable the mode, the config1 of slots event is set to 1. + +On the previous platforms, for user programs using RDPMC directly, the +counter has to be reset explicitly using ioctl: ioctl(perf_fd, PERF_EVENT_IOC_RESET, 0);
Signature · 1 line
2.35.1
Ian Rogers
reviewerRe: [PATCH V2 07/13] perf/x86/intel: Support PERFEVTSEL extension
In reply to [email protected]Quoted 27 lines · On Tue, Jun 25, 2024 at 11:22 AM <[email protected]> wrote:
From: Kan Liang <[email protected]>
Two new fields (the unit mask2, and the equal flag) are added in the
IA32_PERFEVTSELxMSRs. They can be enumerated by the CPUID.23H.0.EBX.Update the
config_maskinx86_pmuandx86_hybrid_pmufor the true layout of the PERFEVTSEL.
Expose the new formats into sysfs if they are available. The umask extension reuses the same format attr name "umask" as the previous umask. Addumask2_showto determine/display the correct format for the current machine.Reviewed-by Andi Kleen <[email protected]>Co-developed-by Dapeng Mi <[email protected]>Signed-off-by Dapeng Mi <[email protected]>Signed-off-by Kan Liang <[email protected]>--- arch/x86/events/intel/core.c | 69 +++++++++++++++++++++++++++++-- arch/x86/include/asm/perf_event.h | 4 ++ 2 files changed, 69 insertions(+), 4 deletions(-)arch/x86/events/intel/core.c +0
-0
arch/x86/events/intel/core.c PMU_FORMAT_ATTR(pc, "config:19" ); · line 4636
Quoted 7 lines
arch/x86/events/intel/core.c +2
-2@@ -4632,8 +4632,55 @@ PMU_FORMAT_ATTR(pc, "config:19" );PMU_FORMAT_ATTR (any,"config:21" );/* v3 + */ PMU_FORMAT_ATTR (inv,"config:23" );PMU_FORMAT_ATTR (cmask,"config:24-31" ); -PMU_FORMAT_ATTR (in_tx,"config:32" ); -PMU_FORMAT_ATTR (in_tx_cp,"config:33" ); +PMU_FORMAT_ATTR (in_tx,"config:32" ); +PMU_FORMAT_ATTR (in_tx_cp,"config:33" );
nit: It seems unfortunate these 2 lines change for the sake of spaces before the ')'. Perhaps leave unchanged.
Thanks,
Ian
Quoted 121 lines
arch/x86/events/intel/core.c +63
-2+PMU_FORMAT_ATTR (eq,"config:36" );/* v6 + */ + +static ssize_t umask2_show (struct device *dev, +struct device_attribute *attr, +char *page) +{ +u64 mask =hybrid (dev_get_drvdata (dev), config_mask) &ARCH_PERFMON_EVENTSEL_UMASK2 ; + +if (mask ==ARCH_PERFMON_EVENTSEL_UMASK2 ) +return sprintf (page,"config:8-15,40-47\n" ); + +/* Roll back to the old format if umask2 is not supported. */ +return sprintf (page,"config:8-15\n" ); +} + +static struct device_attribute format_attr_umask2 = +__ATTR (umask,0444 , umask2_show,NULL ); + +static struct attribute *format_evtsel_ext_attrs[] = { + &format_attr_umask2.attr, + &format_attr_eq.attr, +NULL +}; + +static umode_t +evtsel_ext_is_visible (struct kobject *kobj,struct attribute *attr,int i) +{ +struct device *dev =kobj_to_dev (kobj); +u64 mask; + +/* +* The umask and umask2 have different formats but share the +* same attr name. In update mode, the previous value of the +* umask is unconditionally removed before is_visible. If +* umask2 format is not enumerated, it's impossible to roll +* back to the old format. +* Does the check in umask2_show rather than is_visible. +*/ +if (i ==0 ) +return attr->mode; + + mask =hybrid (dev_get_drvdata (dev), config_mask); +if (i ==1 ) +return (mask &ARCH_PERFMON_EVENTSEL_EQ ) ? attr->mode :0 ; + +return 0 ; +}static struct attribute *intel_arch_formats_attr[] = { &format_attr_event.attr,@@ -4786,8 +4833,14 @@ static inline bool intel_pmu_broken_perf_cap(void)static void update_pmu_cap (struct x86_hybrid_pmu *pmu) { -unsigned int sub_bitmaps =cpuid_eax (ARCH_PERFMON_EXT_LEAF ); -unsigned int eax, ebx, ecx, edx; +unsigned int sub_bitmaps, eax, ebx, ecx, edx; + +cpuid (ARCH_PERFMON_EXT_LEAF , &sub_bitmaps, &ebx, &ecx, &edx); + +if (ebx &ARCH_PERFMON_EXT_UMASK2 ) + pmu->config_mask |=ARCH_PERFMON_EVENTSEL_UMASK2 ; +if (ebx &ARCH_PERFMON_EXT_EQ ) + pmu->config_mask |=ARCH_PERFMON_EVENTSEL_EQ ;if (sub_bitmaps &ARCH_PERFMON_NUM_COUNTER_LEAF_BIT ) { cpuid_count(ARCH_PERFMON_EXT_LEAF, ARCH_PERFMON_NUM_COUNTER_LEAF,@@ -5810,6 +5863,12 @@ static struct attribute_group group_format_extra_skl = { .is_visible = exra_is_visible, }; +static struct attribute_group group_format_evtsel_ext = { + .name ="format" , + .attrs = format_evtsel_ext_attrs, + .is_visible = evtsel_ext_is_visible, +}; +static struct attribute_group group_default = { .attrs = intel_pmu_attrs, .is_visible = default_is_visible,@@ -5823,6 +5882,7 @@ static const struct attribute_group *attr_update[] = { &group_caps_lbr, &group_format_extra, &group_format_extra_skl, + &group_format_evtsel_ext, &group_default, NULL, };@@ -6042,6 +6102,7 @@ static const struct attribute_group *hybrid_attr_update[] = { &group_caps_gen, &group_caps_lbr, &hybrid_group_format_extra, + &group_format_evtsel_ext, &group_default, &hybrid_group_cpus, NULL,arch/x86/include/asm/perf_event.h +4
-1@@ -32,6 +32,8 @@#define ARCH_PERFMON_EVENTSEL_INV (1ULL << 23)#define ARCH_PERFMON_EVENTSEL_CMASK 0xFF000000ULL#define ARCH_PERFMON_EVENTSEL_BR_CNTR (1ULL << 35) +#define ARCH_PERFMON_EVENTSEL_EQ (1ULL << 36) +#define ARCH_PERFMON_EVENTSEL_UMASK2 (0xFFULL << 40)#define INTEL_FIXED_BITS_MASK 0xFULL#define INTEL_FIXED_BITS_STRIDE 4@@ -185,6 +187,8 @@ union cpuid10_edx {* detection/enumeration details: */ #define ARCH_PERFMON_EXT_LEAF 0x00000023 +#define ARCH_PERFMON_EXT_UMASK2 0x1 +#define ARCH_PERFMON_EXT_EQ 0x2#define ARCH_PERFMON_NUM_COUNTER_LEAF_BIT 0x1#define ARCH_PERFMON_NUM_COUNTER_LEAF 0x1 --2.35.1
Quoted 37 lines · On Tue, Jun 25, 2024 at 11:22 AM <[email protected]> wrote:
From: Kan Liang <[email protected]>
Changes since V1:
- Add x86/intel_pmu_max_num_pebs/counters/counters_fixed()
- Rename model-specificpebs_latency_datafunctions
- Rename V6 counter MSRsFrom the core PMU' perspective, the Lunar Lake and Arrow Lake are the same, which are similar to the previous generation Meteor Lake. Both are hybrid platforms, with e-core and p-core.
The key differences include:
- The e-core supports 3 new fixed counters
- The p-core supports an updated PEBS Data Source format
- More GP counters (Updated event constraint table)
- New Architectural performance monitoring V6 (New Perfmon MSRs aliasing, umask2, eq).
- New PEBS format V6 (Counters Snapshotting group)
- New RDPMC metrics clear modeThe details for the above new features can be found in the Intel Architecture Instruction Set Extensions and Future Features (052).
https://cdrdv2.intel.com/v1/dl/getContent/671368The counters may not be continuous anymore. Patch 1-2 converts the max number of counters to a mask of counters. The change is a generic change which impacts all X86 platforms.
Patch 3-4 supports all the legacy features on LNL and ARL.
Patch 5-7 supports the new Architectural performance monitoring V6.
Patch 8-11 supports the new PEBS format V6.
Patch 12 supports the new RDPMC metrics clear mode.
Quoted 14 lines
Kan Liang (13):
perf/x86/intel: Support the PEBS event mask
perf/x86: Support counter mask
perf/x86: Add Lunar Lake and Arrow Lake support perf/x86/intel: Rename model-specificpebs_latency_datafunctions perf/x86/intel: Support new data source for Lunar Lake perf/x86: Addconfig_maskto represent EVENTSEL bitmask perf/x86/intel: Support PERFEVTSEL extension perf/x86/intel: Support Perfmon MSRs aliasing perf/x86: Extend event update interface
perf: Extendperf_output_read
perf/x86/intel: Move PEBS event update after the sample output perf/x86/intel: Support PEBS counters snapshotting perf/x86/intel: Support RDPMC metrics clear mode
Fwiw and beside the nit:
It seems some of the behaviors could have tests upon them, would it be possible to add them skipping the test on older and non-Intel architectures?
Thanks,
Ian
Quoted 21 lines
arch/x86/events/amd/core.c | 26 +- arch/x86/events/core.c | 123 +++---- arch/x86/events/intel/core.c | 471 ++++++++++++++++++++------- arch/x86/events/intel/ds.c | 288 +++++++++++++--- arch/x86/events/intel/knc.c | 2 +- arch/x86/events/intel/p4.c | 12 +- arch/x86/events/intel/p6.c | 2 +- arch/x86/events/perf_event.h | 105 +++++- arch/x86/events/perf_event_flags.h | 2 +- arch/x86/events/zhaoxin/core.c | 14 +- arch/x86/include/asm/intel_ds.h | 1 + arch/x86/include/asm/msr-index.h | 6 + arch/x86/include/asm/perf_event.h | 27 ++ include/uapi/linux/perf_event.h | 6 +- kernel/events/core.c | 15 +- tools/perf/Documentation/topdown.txt | 9 +- 16 files changed, 839 insertions(+), 270 deletions(-)--
2.35.1
Quoted 53 lines · Ian Rogers
Quoted 37 lines · On Tue, Jun 25, 2024 at 11:22 AM <[email protected]> wrote:
From: Kan Liang <[email protected]>
Changes since V1:
- Add x86/intel_pmu_max_num_pebs/counters/counters_fixed()
- Rename model-specificpebs_latency_datafunctions
- Rename V6 counter MSRsFrom the core PMU' perspective, the Lunar Lake and Arrow Lake are the same, which are similar to the previous generation Meteor Lake. Both are hybrid platforms, with e-core and p-core.
The key differences include:
- The e-core supports 3 new fixed counters
- The p-core supports an updated PEBS Data Source format
- More GP counters (Updated event constraint table)
- New Architectural performance monitoring V6 (New Perfmon MSRs aliasing, umask2, eq).
- New PEBS format V6 (Counters Snapshotting group)
- New RDPMC metrics clear modeThe details for the above new features can be found in the Intel Architecture Instruction Set Extensions and Future Features (052).
https://cdrdv2.intel.com/v1/dl/getContent/671368The counters may not be continuous anymore. Patch 1-2 converts the max number of counters to a mask of counters. The change is a generic change which impacts all X86 platforms.
Patch 3-4 supports all the legacy features on LNL and ARL.
Patch 5-7 supports the new Architectural performance monitoring V6.
Patch 8-11 supports the new PEBS format V6.
Patch 12 supports the new RDPMC metrics clear mode.
Quoted 14 lines
Kan Liang (13):
perf/x86/intel: Support the PEBS event mask
perf/x86: Support counter mask
perf/x86: Add Lunar Lake and Arrow Lake support perf/x86/intel: Rename model-specificpebs_latency_datafunctions perf/x86/intel: Support new data source for Lunar Lake perf/x86: Addconfig_maskto represent EVENTSEL bitmask perf/x86/intel: Support PERFEVTSEL extension perf/x86/intel: Support Perfmon MSRs aliasing perf/x86: Extend event update interface
perf: Extendperf_output_read
perf/x86/intel: Move PEBS event update after the sample output perf/x86/intel: Support PEBS counters snapshotting perf/x86/intel: Support RDPMC metrics clear mode
Quoted 3 lines
Fwiw and beside the nit:
Reviewed-by Ian Rogers <[email protected]>
Thanks!
Quoted 4 lines
It seems some of the behaviors could have tests upon them, would it be possible to add them skipping the test on older and non-Intel architectures?
I think only the two features (Architectural performance monitoring V6 and the RDPMC metrics clear mode) add new formats, which impacts the ABI.
The perf tool has a "Sysfs PMU tests" case to verify the event format. It creates a fake sysfs mount to verify all the possible format definitions, rather than the real format on a given machine.
I think the test case make sense, since it's impossible to test all the formats. As long as all the possible combinations of the format definition are supported, the perf tool should be good.
The umask becomes non-contiguous in V6. There is already a case in the "Sysfs PMU tests" to cover such format definition.
The current perf test should be good enough to cover the ABI changes.
9: Sysfs PMU tests : 9.1: Parsing with PMU format directory : Ok 9.2: Parsing with PMU event : Ok 9.3: PMU event names : Ok 9.4: PMU name combining : Ok 9.5: PMU name comparison : Ok
Thanks,
Kan
Quoted 26 lines
Thanks,
IanQuoted 21 lines
arch/x86/events/amd/core.c | 26 +- arch/x86/events/core.c | 123 +++---- arch/x86/events/intel/core.c | 471 ++++++++++++++++++++------- arch/x86/events/intel/ds.c | 288 +++++++++++++--- arch/x86/events/intel/knc.c | 2 +- arch/x86/events/intel/p4.c | 12 +- arch/x86/events/intel/p6.c | 2 +- arch/x86/events/perf_event.h | 105 +++++- arch/x86/events/perf_event_flags.h | 2 +- arch/x86/events/zhaoxin/core.c | 14 +- arch/x86/include/asm/intel_ds.h | 1 + arch/x86/include/asm/msr-index.h | 6 + arch/x86/include/asm/perf_event.h | 27 ++ include/uapi/linux/perf_event.h | 6 +- kernel/events/core.c | 15 +- tools/perf/Documentation/topdown.txt | 9 +- 16 files changed, 839 insertions(+), 270 deletions(-)--
2.35.1
Re: [PATCH V2 07/13] perf/x86/intel: Support PERFEVTSEL extension
In reply to Ian RogersQuoted 23 lines · Ian Rogers
Quoted 22 lines · On Tue, Jun 25, 2024 at 11:22 AM <[email protected]> wrote:
From: Kan Liang <[email protected]>
Two new fields (the unit mask2, and the equal flag) are added in the
IA32_PERFEVTSELxMSRs. They can be enumerated by the CPUID.23H.0.EBX.Update the
config_maskinx86_pmuandx86_hybrid_pmufor the true layout of the PERFEVTSEL.
Expose the new formats into sysfs if they are available. The umask extension reuses the same format attr name "umask" as the previous umask. Addumask2_showto determine/display the correct format for the current machine.Reviewed-by Andi Kleen <[email protected]>Co-developed-by Dapeng Mi <[email protected]>Signed-off-by Dapeng Mi <[email protected]>Signed-off-by Kan Liang <[email protected]>--- arch/x86/events/intel/core.c | 69 +++++++++++++++++++++++++++++-- arch/x86/include/asm/perf_event.h | 4 ++ 2 files changed, 69 insertions(+), 4 deletions(-)
Quoted 15 lines
arch/x86/events/intel/core.c PMU_FORMAT_ATTR(pc, "config:19" ); · line 4636
Quoted 12 lines
arch/x86/events/intel/core.c +2
-2@@ -4632,8 +4632,55 @@ PMU_FORMAT_ATTR(pc, "config:19" );PMU_FORMAT_ATTR (any,"config:21" );/* v3 + */ PMU_FORMAT_ATTR (inv,"config:23" );PMU_FORMAT_ATTR (cmask,"config:24-31" ); -PMU_FORMAT_ATTR (in_tx,"config:32" ); -PMU_FORMAT_ATTR (in_tx_cp,"config:33" ); +PMU_FORMAT_ATTR (in_tx,"config:32" ); +PMU_FORMAT_ATTR (in_tx_cp,"config:33" );nit: It seems unfortunate these 2 lines change for the sake of spaces before the ')'. Perhaps leave unchanged.
The two lines don't follow the existing formats. It kind of ugly when reading the code.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/events/intel/core.c#n4569
I just want to make the format consistent to facilitate the future code review.
Thanks,
Kan
Quoted 125 lines
Thanks,
IanQuoted 121 lines
arch/x86/events/intel/core.c +63
-2+PMU_FORMAT_ATTR (eq,"config:36" );/* v6 + */ + +static ssize_t umask2_show (struct device *dev, +struct device_attribute *attr, +char *page) +{ +u64 mask =hybrid (dev_get_drvdata (dev), config_mask) &ARCH_PERFMON_EVENTSEL_UMASK2 ; + +if (mask ==ARCH_PERFMON_EVENTSEL_UMASK2 ) +return sprintf (page,"config:8-15,40-47\n" ); + +/* Roll back to the old format if umask2 is not supported. */ +return sprintf (page,"config:8-15\n" ); +} + +static struct device_attribute format_attr_umask2 = +__ATTR (umask,0444 , umask2_show,NULL ); + +static struct attribute *format_evtsel_ext_attrs[] = { + &format_attr_umask2.attr, + &format_attr_eq.attr, +NULL +}; + +static umode_t +evtsel_ext_is_visible (struct kobject *kobj,struct attribute *attr,int i) +{ +struct device *dev =kobj_to_dev (kobj); +u64 mask; + +/* +* The umask and umask2 have different formats but share the +* same attr name. In update mode, the previous value of the +* umask is unconditionally removed before is_visible. If +* umask2 format is not enumerated, it's impossible to roll +* back to the old format. +* Does the check in umask2_show rather than is_visible. +*/ +if (i ==0 ) +return attr->mode; + + mask =hybrid (dev_get_drvdata (dev), config_mask); +if (i ==1 ) +return (mask &ARCH_PERFMON_EVENTSEL_EQ ) ? attr->mode :0 ; + +return 0 ; +}static struct attribute *intel_arch_formats_attr[] = { &format_attr_event.attr,@@ -4786,8 +4833,14 @@ static inline bool intel_pmu_broken_perf_cap(void)static void update_pmu_cap (struct x86_hybrid_pmu *pmu) { -unsigned int sub_bitmaps =cpuid_eax (ARCH_PERFMON_EXT_LEAF ); -unsigned int eax, ebx, ecx, edx; +unsigned int sub_bitmaps, eax, ebx, ecx, edx; + +cpuid (ARCH_PERFMON_EXT_LEAF , &sub_bitmaps, &ebx, &ecx, &edx); + +if (ebx &ARCH_PERFMON_EXT_UMASK2 ) + pmu->config_mask |=ARCH_PERFMON_EVENTSEL_UMASK2 ; +if (ebx &ARCH_PERFMON_EXT_EQ ) + pmu->config_mask |=ARCH_PERFMON_EVENTSEL_EQ ;if (sub_bitmaps &ARCH_PERFMON_NUM_COUNTER_LEAF_BIT ) { cpuid_count(ARCH_PERFMON_EXT_LEAF, ARCH_PERFMON_NUM_COUNTER_LEAF,@@ -5810,6 +5863,12 @@ static struct attribute_group group_format_extra_skl = { .is_visible = exra_is_visible, }; +static struct attribute_group group_format_evtsel_ext = { + .name ="format" , + .attrs = format_evtsel_ext_attrs, + .is_visible = evtsel_ext_is_visible, +}; +static struct attribute_group group_default = { .attrs = intel_pmu_attrs, .is_visible = default_is_visible,@@ -5823,6 +5882,7 @@ static const struct attribute_group *attr_update[] = { &group_caps_lbr, &group_format_extra, &group_format_extra_skl, + &group_format_evtsel_ext, &group_default, NULL, };@@ -6042,6 +6102,7 @@ static const struct attribute_group *hybrid_attr_update[] = { &group_caps_gen, &group_caps_lbr, &hybrid_group_format_extra, + &group_format_evtsel_ext, &group_default, &hybrid_group_cpus, NULL,arch/x86/include/asm/perf_event.h +4
-1@@ -32,6 +32,8 @@#define ARCH_PERFMON_EVENTSEL_INV (1ULL << 23)#define ARCH_PERFMON_EVENTSEL_CMASK 0xFF000000ULL#define ARCH_PERFMON_EVENTSEL_BR_CNTR (1ULL << 35) +#define ARCH_PERFMON_EVENTSEL_EQ (1ULL << 36) +#define ARCH_PERFMON_EVENTSEL_UMASK2 (0xFFULL << 40)#define INTEL_FIXED_BITS_MASK 0xFULL#define INTEL_FIXED_BITS_STRIDE 4@@ -185,6 +187,8 @@ union cpuid10_edx {* detection/enumeration details: */ #define ARCH_PERFMON_EXT_LEAF 0x00000023 +#define ARCH_PERFMON_EXT_UMASK2 0x1 +#define ARCH_PERFMON_EXT_EQ 0x2#define ARCH_PERFMON_NUM_COUNTER_LEAF_BIT 0x1#define ARCH_PERFMON_NUM_COUNTER_LEAF 0x1 --2.35.1
Ian Rogers
reviewerQuoted 83 lines · Kan
Quoted 53 lines · Ian Rogers
Quoted 37 lines · On Tue, Jun 25, 2024 at 11:22 AM <[email protected]> wrote:
From: Kan Liang <[email protected]>
Changes since V1:
- Add x86/intel_pmu_max_num_pebs/counters/counters_fixed()
- Rename model-specificpebs_latency_datafunctions
- Rename V6 counter MSRsFrom the core PMU' perspective, the Lunar Lake and Arrow Lake are the same, which are similar to the previous generation Meteor Lake. Both are hybrid platforms, with e-core and p-core.
The key differences include:
- The e-core supports 3 new fixed counters
- The p-core supports an updated PEBS Data Source format
- More GP counters (Updated event constraint table)
- New Architectural performance monitoring V6 (New Perfmon MSRs aliasing, umask2, eq).
- New PEBS format V6 (Counters Snapshotting group)
- New RDPMC metrics clear modeThe details for the above new features can be found in the Intel Architecture Instruction Set Extensions and Future Features (052).
https://cdrdv2.intel.com/v1/dl/getContent/671368The counters may not be continuous anymore. Patch 1-2 converts the max number of counters to a mask of counters. The change is a generic change which impacts all X86 platforms.
Patch 3-4 supports all the legacy features on LNL and ARL.
Patch 5-7 supports the new Architectural performance monitoring V6.
Patch 8-11 supports the new PEBS format V6.
Patch 12 supports the new RDPMC metrics clear mode.
Quoted 14 lines
Kan Liang (13):
perf/x86/intel: Support the PEBS event mask
perf/x86: Support counter mask
perf/x86: Add Lunar Lake and Arrow Lake support perf/x86/intel: Rename model-specificpebs_latency_datafunctions perf/x86/intel: Support new data source for Lunar Lake perf/x86: Addconfig_maskto represent EVENTSEL bitmask perf/x86/intel: Support PERFEVTSEL extension perf/x86/intel: Support Perfmon MSRs aliasing perf/x86: Extend event update interface
perf: Extendperf_output_read
perf/x86/intel: Move PEBS event update after the sample output perf/x86/intel: Support PEBS counters snapshotting perf/x86/intel: Support RDPMC metrics clear modeQuoted 3 lines
Fwiw and beside the nit:
Reviewed-by Ian Rogers <[email protected]>Thanks!
Quoted 4 lines
It seems some of the behaviors could have tests upon them, would it be possible to add them skipping the test on older and non-Intel architectures?
I think only the two features (Architectural performance monitoring V6 and the RDPMC metrics clear mode) add new formats, which impacts the ABI.
The perf tool has a "Sysfs PMU tests" case to verify the event format. It creates a fake sysfs mount to verify all the possible format definitions, rather than the real format on a given machine.
I think the test case make sense, since it's impossible to test all the formats. As long as all the possible combinations of the format definition are supported, the perf tool should be good.
The umask becomes non-contiguous in V6. There is already a case in the "Sysfs PMU tests" to cover such format definition.
The current perf test should be good enough to cover the ABI changes.
Quoted 6 lines
9: Sysfs PMU tests : 9.1: Parsing with PMU format directory : Ok 9.2: Parsing with PMU event : Ok 9.3: PMU event names : Ok 9.4: PMU name combining : Ok 9.5: PMU name comparison : Ok
That's good. I was thinking more about leader sampling testing. We have an attribute test and an event parsing test, but nothing that currently tests leader sampling actually works even if it is just a coverage test. We could do something like this:
```
diff --git a/tools/perf/tests/shell/record.sh b/tools/perf/tests/shell/record.sh
index 3d1a7759a7b2..9b61e2d01c9e 100755
--- a/tools/perf/tests/shell/record.sh
+++ b/tools/perf/tests/shell/record.sh
@@ -190,11 +190,29 @@ test_branch_counter() {
echo "Basic branch counter test [Success]"
}
+test_leader_sampling() {
+ echo "Basic leader sampling test"
+ if ! perf record -o "${perfdata}" -g -e
"{instructions,branch-misses}:S" ${testprog} 2> /dev/null
+ then
+ echo "Leader sampling [Failed record]"
+ err=1
+ return
+ fi
+ if ! perf script -D -i "${perfdata}" | grep -q "sample_read:"
+ then
+ echo "Leader sampling [Failed missing output]"
+ err=1
+ return
+ fi
+ echo "Basic leader sampling test [Success]"
+}
+
test_per_thread
test_register_capture
test_system_wide
test_workload
test_branch_counter
+test_leader_sampling
cleanup
exit $err
```But it'd be nice if we had coverage of things like fixed counters vs generic, the improvements now fixed in LNL, etc.
Thanks,
Ian
Quoted 28 lines
Thanks,
KanQuoted 26 lines
Thanks,
IanQuoted 21 lines
arch/x86/events/amd/core.c | 26 +- arch/x86/events/core.c | 123 +++---- arch/x86/events/intel/core.c | 471 ++++++++++++++++++++------- arch/x86/events/intel/ds.c | 288 +++++++++++++--- arch/x86/events/intel/knc.c | 2 +- arch/x86/events/intel/p4.c | 12 +- arch/x86/events/intel/p6.c | 2 +- arch/x86/events/perf_event.h | 105 +++++- arch/x86/events/perf_event_flags.h | 2 +- arch/x86/events/zhaoxin/core.c | 14 +- arch/x86/include/asm/intel_ds.h | 1 + arch/x86/include/asm/msr-index.h | 6 + arch/x86/include/asm/perf_event.h | 27 ++ include/uapi/linux/perf_event.h | 6 +- kernel/events/core.c | 15 +- tools/perf/Documentation/topdown.txt | 9 +- 16 files changed, 839 insertions(+), 270 deletions(-)--
2.35.1
Ian Rogers
reviewerRe: [PATCH V2 07/13] perf/x86/intel: Support PERFEVTSEL extension
In reply to Liang, KanQuoted 43 lines · Kan
Quoted 23 lines · Ian Rogers
Quoted 22 lines · On Tue, Jun 25, 2024 at 11:22 AM <[email protected]> wrote:
From: Kan Liang <[email protected]>
Two new fields (the unit mask2, and the equal flag) are added in the
IA32_PERFEVTSELxMSRs. They can be enumerated by the CPUID.23H.0.EBX.Update the
config_maskinx86_pmuandx86_hybrid_pmufor the true layout of the PERFEVTSEL.
Expose the new formats into sysfs if they are available. The umask extension reuses the same format attr name "umask" as the previous umask. Addumask2_showto determine/display the correct format for the current machine.Reviewed-by Andi Kleen <[email protected]>Co-developed-by Dapeng Mi <[email protected]>Signed-off-by Dapeng Mi <[email protected]>Signed-off-by Kan Liang <[email protected]>--- arch/x86/events/intel/core.c | 69 +++++++++++++++++++++++++++++-- arch/x86/include/asm/perf_event.h | 4 ++ 2 files changed, 69 insertions(+), 4 deletions(-)Quoted 15 lines
arch/x86/events/intel/core.c PMU_FORMAT_ATTR(pc, "config:19" ); · line 4636
Quoted 12 lines
arch/x86/events/intel/core.c +2
-2@@ -4632,8 +4632,55 @@ PMU_FORMAT_ATTR(pc, "config:19" );PMU_FORMAT_ATTR (any,"config:21" );/* v3 + */ PMU_FORMAT_ATTR (inv,"config:23" );PMU_FORMAT_ATTR (cmask,"config:24-31" ); -PMU_FORMAT_ATTR (in_tx,"config:32" ); -PMU_FORMAT_ATTR (in_tx_cp,"config:33" ); +PMU_FORMAT_ATTR (in_tx,"config:32" ); +PMU_FORMAT_ATTR (in_tx_cp,"config:33" );nit: It seems unfortunate these 2 lines change for the sake of spaces before the ')'. Perhaps leave unchanged.
Quoted 6 lines
The two lines don't follow the existing formats. It kind of ugly when reading the code.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/events/intel/core.c#n4569I just want to make the format consistent to facilitate the future code review.
Thanks, I'm easy on just keeping it the same or changing the whitespace. When I saw the lines changing I was figuring the config encoding was being modified which could break things like libpfm4 that directly use configs rather than formats.
Thanks,
Ian
Quoted 91 lines · Ian Rogers
Quoted 83 lines · Kan
Quoted 53 lines · Ian Rogers
Quoted 37 lines · On Tue, Jun 25, 2024 at 11:22 AM <[email protected]> wrote:
From: Kan Liang <[email protected]>
Changes since V1:
- Add x86/intel_pmu_max_num_pebs/counters/counters_fixed()
- Rename model-specificpebs_latency_datafunctions
- Rename V6 counter MSRsFrom the core PMU' perspective, the Lunar Lake and Arrow Lake are the same, which are similar to the previous generation Meteor Lake. Both are hybrid platforms, with e-core and p-core.
The key differences include:
- The e-core supports 3 new fixed counters
- The p-core supports an updated PEBS Data Source format
- More GP counters (Updated event constraint table)
- New Architectural performance monitoring V6 (New Perfmon MSRs aliasing, umask2, eq).
- New PEBS format V6 (Counters Snapshotting group)
- New RDPMC metrics clear modeThe details for the above new features can be found in the Intel Architecture Instruction Set Extensions and Future Features (052).
https://cdrdv2.intel.com/v1/dl/getContent/671368The counters may not be continuous anymore. Patch 1-2 converts the max number of counters to a mask of counters. The change is a generic change which impacts all X86 platforms.
Patch 3-4 supports all the legacy features on LNL and ARL.
Patch 5-7 supports the new Architectural performance monitoring V6.
Patch 8-11 supports the new PEBS format V6.
Patch 12 supports the new RDPMC metrics clear mode.
Quoted 14 lines
Kan Liang (13):
perf/x86/intel: Support the PEBS event mask
perf/x86: Support counter mask
perf/x86: Add Lunar Lake and Arrow Lake support perf/x86/intel: Rename model-specificpebs_latency_datafunctions perf/x86/intel: Support new data source for Lunar Lake perf/x86: Addconfig_maskto represent EVENTSEL bitmask perf/x86/intel: Support PERFEVTSEL extension perf/x86/intel: Support Perfmon MSRs aliasing perf/x86: Extend event update interface
perf: Extendperf_output_read
perf/x86/intel: Move PEBS event update after the sample output perf/x86/intel: Support PEBS counters snapshotting perf/x86/intel: Support RDPMC metrics clear modeQuoted 3 lines
Fwiw and beside the nit:
Reviewed-by Ian Rogers <[email protected]>Thanks!
Quoted 4 lines
It seems some of the behaviors could have tests upon them, would it be possible to add them skipping the test on older and non-Intel architectures?
I think only the two features (Architectural performance monitoring V6 and the RDPMC metrics clear mode) add new formats, which impacts the ABI.
The perf tool has a "Sysfs PMU tests" case to verify the event format. It creates a fake sysfs mount to verify all the possible format definitions, rather than the real format on a given machine.
I think the test case make sense, since it's impossible to test all the formats. As long as all the possible combinations of the format definition are supported, the perf tool should be good.
The umask becomes non-contiguous in V6. There is already a case in the "Sysfs PMU tests" to cover such format definition.
The current perf test should be good enough to cover the ABI changes.Quoted 6 lines
9: Sysfs PMU tests : 9.1: Parsing with PMU format directory : Ok 9.2: Parsing with PMU event : Ok 9.3: PMU event names : Ok 9.4: PMU name combining : Ok 9.5: PMU name comparison : Ok
Quoted 4 lines
That's good. I was thinking more about leader sampling testing. We have an attribute test and an event parsing test, but nothing that currently tests leader sampling actually works even if it is just a coverage test.
Yes, the sampling read test is missed.
Quoted 36 lines
We could do something like this:
```
tools/perf/tests/shell/record.sh +3
-0@@ -190,11 +190,29 @@ test_branch_counter() { echo "Basic branch counter test [Success]" } +test_leader_sampling() { + echo "Basic leader sampling test" + if ! perf record -o "${perfdata}" -g -e"{instructions,branch-misses}:S" ${testprog} 2>
/dev/nulltools/perf/tests/shell/record.sh +14
-0+ then + echo "Leader sampling [Failed record]" + err=1 + return + fi + if ! perf script -D -i "${perfdata}" | grep -q "sample_read:" + then + echo "Leader sampling [Failed missing output]" + err=1 + return + fi + echo "Basic leader sampling test [Success]" +} +
test_per_threadtest_register_capturetest_system_widetest_workloadtest_branch_counter
+test_leader_sampling
Quoted 6 lines
cleanup
exit $err
```But it'd be nice if we had coverage of things like fixed counters vs generic, the improvements now fixed in LNL, etc.
The improvements in LNL should be transparent for the perf tool. It just changes the places where to read the counter (from "directly read the HW counter" to "read it in a PEBS record"). It is hard to design a perf tool test case to verify the small difference.
But, I guess we can choose proper events to test the fixed counters vs generic.
Also, there are some special cases for the sampling read. For example, the AUX area event and the Topdown slots event cannot be sampled. If they are the leader, the next event will be the sampling event. We may also design a case to cover it as well.
I will think about it and send a separate patch to cover the sampling read.
Thanks,
Kan
Quoted 32 lines
Thanks,
IanQuoted 28 lines
Thanks,
KanQuoted 26 lines
Thanks,
IanQuoted 21 lines
arch/x86/events/amd/core.c | 26 +- arch/x86/events/core.c | 123 +++---- arch/x86/events/intel/core.c | 471 ++++++++++++++++++++------- arch/x86/events/intel/ds.c | 288 +++++++++++++--- arch/x86/events/intel/knc.c | 2 +- arch/x86/events/intel/p4.c | 12 +- arch/x86/events/intel/p6.c | 2 +- arch/x86/events/perf_event.h | 105 +++++- arch/x86/events/perf_event_flags.h | 2 +- arch/x86/events/zhaoxin/core.c | 14 +- arch/x86/include/asm/intel_ds.h | 1 + arch/x86/include/asm/msr-index.h | 6 + arch/x86/include/asm/perf_event.h | 27 ++ include/uapi/linux/perf_event.h | 6 +- kernel/events/core.c | 15 +- tools/perf/Documentation/topdown.txt | 9 +- 16 files changed, 839 insertions(+), 270 deletions(-)--
2.35.1