perf/x86/amd: Add AMD LbrExtV2 support
Subsystems (2)
cover perf/x86/amd: Add AMD LbrExtV2 support
Last Branch Record (LBR) is a feature available on modern processors for recording branch information. It helps determine the flow of control by logging branch information to registers in realtime and helps with the detection of hot code paths.
Add support for using AMD Last Branch Record Extension Version 2 (LbrExtV2) features on Zen 4 processors. New CPU features are introduced for LbrExtV2 detection. New MSR definitions are added for configuring hardware branch filtering and for enabling the LBR Freeze on PMI feature.
The LBR Freeze on PMI feature is essential for ensuring that branch records remain consistent with the point of PMU overflow in order to provide a precise correlation between the two.
Hardware branch filtering allows users to record only specific types of branches and can be mapped to most of the existing filters supported by the perf tool. Additional software filtering ensures that some special branches (syscall entry and exit) for which direct hardware filters do not exist are also recorded. This expands the scope of filters like "any_call".
Additionally, the perf UAPI is now extended to provide branch speculation information, if available. LbrExtV2 provides this information through the "valid" and "spec" bits in the Branch To registers. The tools-side changes for this will be submitted as a separate series.
Users of perf tool can now record branches as shown below. The 'div' workload used here is from https://lwn.net/Articles/680985/.
E.g.
$ perf record -b -e cycles:u ./div
Before:
Error:
cycles:u: PMU Hardware doesn't support sampling/overflow-interrupts. Try 'perf stat'
After:
[ perf record: Woken up 49 times to write data ]
[ perf record: Captured and wrote 12.197 MB perf.data (29601 samples) ]
$ perf report --stdio
# To display the perf.data header info, please use --header/--header-only options. #
#
# Total Lost Samples: 0
#
# Samples: 473K of event 'cycles:u'
# Event count (approx.): 473521
#
# Overhead Command Source Shared Object Source Symbol Target Symbol Basic Block Cycles
# ........ ....... .................... ...................... ...................... ..................
#
29.69% div div [.] main [.] main -
23.84% div div [.] compute_flag [.] main -
23.41% div div [.] compute_flag [.] compute_flag -
23.04% div div [.] main [.] compute_flag -[...]
No additional failures are seen upon running the following:
* perf built-in test suite
* perf_event_tests suite
Sandipan Das (13):
perf/x86/amd/brs: Move feature-specific functions perf/x86/amd/core: Refactor branch attributes perf/x86/amd/core: Add generic branch record interfaces x86/cpufeatures: Add LbrExtV2 feature bit perf/x86/amd/lbr: Detect LbrExtV2 support perf/x86/amd/lbr: Add LbrExtV2 branch record support perf/x86/amd/lbr: Add LbrExtV2 hardware branch filter support perf/x86: Move branch classifier
perf/x86/amd/lbr: Add LbrExtV2 software branch filter support perf/x86: Make branch classifier fusion-aware perf/x86/amd/lbr: Use fusion-aware branch classifier perf/core: Add speculation info to branch entries perf/x86/amd/lbr: Add LbrExtV2 branch speculation info support
arch/x86/events/Makefile | 2 +- arch/x86/events/amd/Makefile | 2 +- arch/x86/events/amd/brs.c | 69 ++++- arch/x86/events/amd/core.c | 200 +++++++------ arch/x86/events/amd/lbr.c | 435 +++++++++++++++++++++++++++++ arch/x86/events/intel/lbr.c | 273 ------------------ arch/x86/events/perf_event.h | 81 +++++- arch/x86/events/utils.c | 247 ++++++++++++++++ arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/include/asm/msr-index.h | 5 + arch/x86/include/asm/perf_event.h | 3 +- arch/x86/kernel/cpu/scattered.c | 1 + include/linux/perf_event.h | 1 + include/uapi/linux/perf_event.h | 15 +- 14 files changed, 952 insertions(+), 384 deletions(-) create mode 100644 arch/x86/events/amd/lbr.c create mode 100644 arch/x86/events/utils.c
Signature · 1 line
2.34.1
PATCH 1/13 perf/x86/amd/brs: Move feature-specific functions
Move some of the Branch Sampling (BRS) specific functions out of the Core events sources and into the BRS sources in preparation for adding other mechanisms to record branches.
--- arch/x86/events/amd/brs.c | 69 ++++++++++++++++++++++++++++++++++- arch/x86/events/amd/core.c | 70 ++---------------------------------- arch/x86/events/perf_event.h | 7 ++-- 3 files changed, 76 insertions(+), 70 deletions(-)
arch/x86/events/amd/brs.c +68 -1
@@ -81,7 +81,7 @@ static bool __init amd_brs_detect(void)* a br_sel_map. Software filtering is not supported because it would not correlate well * with a sampling period. */ -int amd_brs_setup_filter (struct perf_event *event) +static int amd_brs_setup_filter (struct perf_event *event) {u64 type = event->attr.branch_sample_type;
@@ -96,6 +96,73 @@ int amd_brs_setup_filter(struct perf_event *event)return 0 ; } +static inline int amd_is_brs_event (struct perf_event *e) +{ +return (e->hw.config &AMD64_RAW_EVENT_MASK ) ==AMD_FAM19H_BRS_EVENT ; +} + +int amd_brs_hw_config (struct perf_event *event) +{ +int ret =0 ; + +/* +* Due to interrupt holding, BRS is not recommended in +* counting mode. +*/ +if (!is_sampling_event (event)) +return -EINVAL ; + +/* +* Due to the way BRS operates by holding the interrupt until +* lbr_nr entries have been captured, it does not make sense +* to allow sampling on BRS with an event that does not match +* what BRS is capturing, i.e., retired taken branches. +* Otherwise the correlation with the event's period is even +* more loose: +* +* With retired taken branch: +* Effective P = P + 16 + X +* With any other event: +* Effective P = P + Y + X +* +* Where X is the number of taken branches due to interrupt +* skid. Skid is large. +* +* Where Y is the occurences of the event while BRS is +* capturing the lbr_nr entries. +* +* By using retired taken branches, we limit the impact on the +* Y variable. We know it cannot be more than the depth of +* BRS. +*/ +if (!amd_is_brs_event (event)) +return -EINVAL ; + +/* +* BRS implementation does not work with frequency mode +* reprogramming of the period. +*/ +if (event->attr.freq) +return -EINVAL ; +/* +* The kernel subtracts BRS depth from period, so it must +* be big enough. +*/ +if (event->attr.sample_period <= x86_pmu.lbr_nr) +return -EINVAL ; + +/* +* Check if we can allow PERF_SAMPLE_BRANCH_STACK +*/ + ret =amd_brs_setup_filter (event); + +/* only set in case of success */ +if (!ret) + event->hw.flags |=PERF_X86_EVENT_AMD_BRS ; + +return ret; +} +/* tos = top of stack, i.e., last valid entry written */ static inline int amd_brs_get_tos (union amd_debug_extn_cfg *cfg) {
arch/x86/events/amd/core.c +3 -67
85 lines not on this page · raw message
arch/x86/events/perf_event.h +5 -2
@@ -1231,6 +1231,9 @@ static inline bool fixed_counter_disabled(int i, struct pmu *pmu)int amd_pmu_init (void );#ifdef CONFIG_PERF_EVENTS_AMD_BRS + +#define AMD_FAM19H_BRS_EVENT 0xc4/* RETIRED_TAKEN_BRANCH_INSTRUCTIONS */ +int amd_brs_init (void );void amd_brs_disable (void );void amd_brs_enable (void );
@@ -1239,7 +1242,7 @@ void amd_brs_disable_all(void);void amd_brs_drain (void );void amd_brs_lopwr_init (void );void amd_brs_disable_all (void ); -int amd_brs_setup_filter (struct perf_event *event); +int amd_brs_hw_config (struct perf_event *event);void amd_brs_reset (void );static inline void amd_pmu_brs_add (struct perf_event *event)
@@ -1275,7 +1278,7 @@ static inline void amd_brs_enable(void) {}static inline void amd_brs_drain (void ) {}static inline void amd_brs_lopwr_init (void ) {}static inline void amd_brs_disable_all (void ) {} -static inline int amd_brs_setup_filter (struct perf_event *event) +static inline int amd_brs_hw_config (struct perf_event *event) {return 0 ; }
Signature · 1 line
2.34.1
PATCH 2/13 perf/x86/amd/core: Refactor branch attributes
In reply to Sandipan DasAMD processors that are capable of recording branches support either Branch Sampling (BRS) or Last Branch Record (LBR). In preparation for adding Last Branch Record Extension Version 2 (LbrExtV2) support, reuse the "branches" capability to advertise information about both BRS and LBR but make the "branch-brs" event exclusive to Family 19h processors that support BRS.
--- arch/x86/events/amd/core.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-)
arch/x86/events/amd/core.c +19 -6
@@ -1247,23 +1247,25 @@ static ssize_t branches_show(struct device *cdev,static DEVICE_ATTR_RO (branches); -static struct attribute *amd_pmu_brs_attrs[] = { +static struct attribute *amd_pmu_branches_attrs[] = { &dev_attr_branches.attr,NULL , };static umode_t -amd_brs_is_visible (struct kobject *kobj,struct attribute *attr,int i) +amd_branches_is_visible (struct kobject *kobj,struct attribute *attr,int i) {return x86_pmu.lbr_nr ? attr->mode :0 ; } -static struct attribute_group group_caps_amd_brs = { +static struct attribute_group group_caps_amd_branches = { .name ="caps" , - .attrs = amd_pmu_brs_attrs, - .is_visible = amd_brs_is_visible, + .attrs = amd_pmu_branches_attrs, + .is_visible = amd_branches_is_visible, }; +#ifdef CONFIG_PERF_EVENTS_AMD_BRS +EVENT_ATTR_STR (branch-brs, amd_branch_brs, "event=" __stringify(AMD_FAM19H_BRS_EVENT)"\n");
@@ -1272,15 +1274,26 @@ static struct attribute *amd_brs_events_attrs[] = { NULL, }; +static umode_t +amd_brs_is_visible (struct kobject *kobj,struct attribute *attr,int i) +{ +return static_cpu_has (X86_FEATURE_BRS ) && x86_pmu.lbr_nr ? + attr->mode :0 ; +} +static struct attribute_group group_events_amd_brs = { .name ="events" , .attrs = amd_brs_events_attrs, .is_visible = amd_brs_is_visible, }; +#endif /* CONFIG_PERF_EVENTS_AMD_BRS */ +static const struct attribute_group *amd_attr_update[] = { - &group_caps_amd_brs, + &group_caps_amd_branches, +#ifdef CONFIG_PERF_EVENTS_AMD_BRS &group_events_amd_brs, +#endifNULL , };
Signature · 1 line
2.34.1
PATCH 3/13 perf/x86/amd/core: Add generic branch record interfaces
In reply to Sandipan DasAMD processors that are capable of recording branches support either Branch Sampling (BRS) or Last Branch Record (LBR). In preparation for adding Last Branch Record Extension Version 2 (LbrExtV2) support, introduce new static calls which act as gateways to call into the feature-dependent functions based on what is available on the processor.
--- arch/x86/events/amd/core.c | 34 ++++++++++++++++++++++------------ 1 file changed, 22 insertions(+), 12 deletions(-)
arch/x86/events/amd/core.c +22 -12
@@ -330,6 +330,8 @@ static inline bool amd_is_pair_event_code(struct hw_perf_event *hwc) } } +DEFINE_STATIC_CALL_RET0 (amd_pmu_branch_hw_config, *x86_pmu.hw_config); +static int amd_core_hw_config (struct perf_event *event) {if (event->attr.exclude_host && event->attr.exclude_guest)
@@ -349,7 +351,7 @@ static int amd_core_hw_config(struct perf_event *event) event->hw.flags |=PERF_X86_EVENT_PAIR ;if (has_branch_stack (event)) -return amd_brs_hw_config (event); +return static_call (amd_pmu_branch_hw_config)(event);return 0 ; }
@@ -518,8 +520,14 @@ static struct amd_nb *amd_alloc_nb(int cpu)return nb; } +typedef void (amd_pmu_branch_reset_t )(void ); +DEFINE_STATIC_CALL_NULL (amd_pmu_branch_reset, amd_pmu_branch_reset_t); +static void amd_pmu_cpu_reset (int cpu) { +if (x86_pmu.lbr_nr) +static_call (amd_pmu_branch_reset)(); +if (x86_pmu.version <2 )return ;
@@ -576,7 +584,6 @@ static void amd_pmu_cpu_starting(int cpu) cpuc->amd_nb->nb_id = nb_id; cpuc->amd_nb->refcnt++; -amd_brs_reset ();amd_pmu_cpu_reset (cpu); }
@@ -771,16 +778,20 @@ static void amd_pmu_v2_disable_all(void)amd_pmu_check_overflow (); } +DEFINE_STATIC_CALL_NULL (amd_pmu_branch_add, *x86_pmu.add); +static void amd_pmu_add_event (struct perf_event *event) {if (needs_branch_stack (event)) -amd_pmu_brs_add (event); +static_call (amd_pmu_branch_add)(event); } +DEFINE_STATIC_CALL_NULL (amd_pmu_branch_del, *x86_pmu.del); +static void amd_pmu_del_event (struct perf_event *event) {if (needs_branch_stack (event)) -amd_pmu_brs_del (event); +static_call (amd_pmu_branch_del)(event); }/*
@@ -1184,13 +1195,6 @@ static ssize_t amd_event_sysfs_show(char *page, u64 config)return x86_event_sysfs_show (page, config, event); } -static void amd_pmu_sched_task (struct perf_event_context *ctx, -bool sched_in) -{ -if (sched_in && x86_pmu.lbr_nr) -amd_pmu_brs_sched_task (ctx, sched_in); -} -static u64 amd_pmu_limit_period (struct perf_event *event,u64 left) {/*
@@ -1375,8 +1379,14 @@ static int __init amd_core_pmu_init(void)*/ if (boot_cpu_data.x86 >=0x19 && !amd_brs_init ()) { x86_pmu.get_event_constraints = amd_get_event_constraints_f19h; - x86_pmu.sched_task = amd_pmu_sched_task; + x86_pmu.sched_task = amd_pmu_brs_sched_task; x86_pmu.limit_period = amd_pmu_limit_period; + +static_call_update (amd_pmu_branch_hw_config, amd_brs_hw_config); +static_call_update (amd_pmu_branch_reset, amd_brs_reset); +static_call_update (amd_pmu_branch_add, amd_pmu_brs_add); +static_call_update (amd_pmu_branch_del, amd_pmu_brs_del); +/* * put_event_constraints callback same as Fam17h, set above */
Signature · 1 line
2.34.1
PATCH 4/13 x86/cpufeatures: Add LbrExtV2 feature bit
In reply to Sandipan DasCPUID leaf 0x80000022 i.e. ExtPerfMonAndDbg advertises some new performance monitoring features for AMD processors.
Bit 1 of EAX indicates support for Last Branch Record Extension Version 2 (LbrExtV2) features. If found to be set during PMU initialization, the EBX bits of the same leaf can be used to determine the number of available LBR entries.
For better utilization of feature words, LbrExtV2 is added as a scattered feature bit.
--- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpufeatures.h +1 -1
@@ -96,7 +96,7 @@#define X86_FEATURE_SYSCALL32 ( 3*32+14)/* "" syscall in IA32 userspace */ #define X86_FEATURE_SYSENTER32 ( 3*32+15)/* "" sysenter in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16)/* REP microcode works well */ -/* FREE! ( 3*32+17) */ +#define X86_FEATURE_LBREXT_V2 ( 3*32+17)/* AMD Last Branch Record Extension Version 2 */ #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18)/* "" LFENCE synchronizes RDTSC */ #define X86_FEATURE_ACC_POWER ( 3*32+19)/* AMD Accumulated Power Mechanism */ #define X86_FEATURE_NOPL ( 3*32+20)/* The NOPL (0F 1F) instructions */
arch/x86/kernel/cpu/scattered.c +1 -0
@@ -44,6 +44,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, { X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 }, + { X86_FEATURE_LBREXT_V2, CPUID_EAX, 1, 0x80000022, 0 }, { 0, 0, 0, 0, 0 } };
Signature · 1 line
2.34.1
PATCH 5/13 perf/x86/amd/lbr: Detect LbrExtV2 support
In reply to Sandipan DasAMD Last Branch Record Extension Version 2 (LbrExtV2) is driven by Core PMC overflows. It records recently taken branches up to the moment when the PMC overflow occurs.
Detect the feature during PMU initialization and set the branch stack depth using CPUID leaf 0x80000022 EBX.
--- arch/x86/events/amd/Makefile | 2 +- arch/x86/events/amd/core.c | 9 +++++---- arch/x86/events/amd/lbr.c | 21 +++++++++++++++++++++ arch/x86/events/perf_event.h | 2 ++ arch/x86/include/asm/perf_event.h | 3 ++- 5 files changed, 31 insertions(+), 6 deletions(-) create mode 100644 arch/x86/events/amd/lbr.c
arch/x86/events/amd/Makefile +1 -1
@@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_CPU_SUP_AMD) += core.o +obj-$(CONFIG_CPU_SUP_AMD) += core.o lbr.o obj-$(CONFIG_PERF_EVENTS_AMD_BRS) += brs.o obj-$(CONFIG_PERF_EVENTS_AMD_POWER) += power.o obj-$(CONFIG_X86_LOCAL_APIC) += ibs.o
arch/x86/events/amd/core.c +5 -4
@@ -1374,10 +1374,11 @@ static int __init amd_core_pmu_init(void) x86_pmu.flags |=PMU_FL_PAIR ; } -/* -* BRS requires special event constraints and flushing on ctxsw. -*/ -if (boot_cpu_data.x86 >=0x19 && !amd_brs_init ()) { +/* LBR and BRS are mutually exclusive features */ +if (amd_pmu_lbr_init () && !amd_brs_init ()) { +/* +* BRS requires special event constraints and flushing on ctxsw. +*/ x86_pmu.get_event_constraints = amd_get_event_constraints_f19h; x86_pmu.sched_task = amd_pmu_brs_sched_task; x86_pmu.limit_period = amd_pmu_limit_period;
arch/x86/events/amd/lbr.c new file +21 -0
@@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/perf_event.h> +#include <asm/perf_event.h> + +#include "../perf_event.h" + +__init int amd_pmu_lbr_init (void ) +{ +union cpuid_0x80000022_ebx ebx; + +if (x86_pmu.version <2 || !boot_cpu_has (X86_FEATURE_LBREXT_V2 )) +return -EOPNOTSUPP ; + +/* Set number of entries */ + ebx.full =cpuid_ebx (EXT_PERFMON_DEBUG_FEATURES ); + x86_pmu.lbr_nr = ebx.split.lbr_v2_stack_sz; + +pr_cont ("%d-deep LBR, " , x86_pmu.lbr_nr); + +return 0 ; +}
arch/x86/events/perf_event.h +2 -0
@@ -1230,6 +1230,8 @@ static inline bool fixed_counter_disabled(int i, struct pmu *pmu)int amd_pmu_init (void ); +int amd_pmu_lbr_init (void ); +#ifdef CONFIG_PERF_EVENTS_AMD_BRS #define AMD_FAM19H_BRS_EVENT 0xc4/* RETIRED_TAKEN_BRANCH_INSTRUCTIONS */
arch/x86/include/asm/perf_event.h +2 -1
@@ -207,7 +207,8 @@ union cpuid_0x80000022_ebx {struct {/* Number of Core Performance Counters */ unsigned int num_core_pmc:4 ; -unsigned int reserved:6 ; +/* Number of available LBR Stack Entries */ +unsigned int lbr_v2_stack_sz:6 ;/* Number of Data Fabric Counters */ unsigned int num_df_pmc:6 ; } split;
Signature · 1 line
2.34.1
PATCH 6/13 perf/x86/amd/lbr: Add LbrExtV2 branch record support
In reply to Sandipan DasIf AMD Last Branch Record Extension Version 2 (LbrExtV2) is detected, enable it alongside LBR Freeze on PMI when an event requests branch stack i.e. PERF_SAMPLE_BRANCH_STACK.
Each branch record is represented by a pair of registers, LBR From and LBR To. The freeze feature prevents any updates to these registers once a PMC overflows. The contents remain unchanged until the freeze bit is cleared by the PMI handler.
The branch records are read and copied to sample data before unfreezing. However, only valid entries are copied. There is no additional register to denote which of the register pairs represent the top of the stack (TOS) since internal register renaming always ensures that the first pair (i.e. index 0) is the one representing the most recent branch and so on.
The LBR registers are per-thread resources and are cleared explicitly whenever a new task is scheduled in. There are no special implications on the contents of these registers when transitioning to deep C-states.
--- arch/x86/events/amd/core.c | 47 +++++-- arch/x86/events/amd/lbr.c | 203 +++++++++++++++++++++++++++++++ arch/x86/events/perf_event.h | 8 ++ arch/x86/include/asm/msr-index.h | 5 + 4 files changed, 252 insertions(+), 11 deletions(-)
arch/x86/events/amd/core.c +36 -11
@@ -620,7 +620,7 @@ static inline u64 amd_pmu_get_global_status(void)/* PerfCntrGlobalStatus is read-only */ rdmsrl (MSR_AMD64_PERF_CNTR_GLOBAL_STATUS , status); -return status & amd_pmu_global_cntr_mask; +return status; }static inline void amd_pmu_ack_global_status (u64 status)
@@ -631,8 +631,6 @@ static inline void amd_pmu_ack_global_status(u64 status)* clears the same bit in PerfCntrGlobalStatus */ -/* Only allow modifications to PerfCntrGlobalStatus.PerfCntrOvfl */ - status &= amd_pmu_global_cntr_mask;wrmsrl (MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR , status); }
@@ -742,11 +740,17 @@ static void amd_pmu_v2_enable_event(struct perf_event *event)__x86_pmu_enable_event (hwc,ARCH_PERFMON_EVENTSEL_ENABLE ); } -static void amd_pmu_v2_enable_all (int added) +static __always_inline void amd_pmu_core_enable_all(void) {amd_pmu_set_global_ctl (amd_pmu_global_cntr_mask); } +static void amd_pmu_v2_enable_all (int added) +{ +amd_pmu_lbr_enable_all (); +amd_pmu_core_enable_all (); +} +static void amd_pmu_disable_event (struct perf_event *event) {x86_pmu_disable_event (event);
@@ -771,10 +775,15 @@ static void amd_pmu_disable_all(void)amd_pmu_check_overflow (); } -static void amd_pmu_v2_disable_all (void ) +static __always_inline void amd_pmu_core_disable_all(void) { -/* Disable all PMCs */ amd_pmu_set_global_ctl (0 ); +} + +static void amd_pmu_v2_disable_all (void ) +{ +amd_pmu_core_disable_all (); +amd_pmu_lbr_disable_all ();amd_pmu_check_overflow (); }
@@ -877,8 +886,8 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) pmu_enabled = cpuc->enabled; cpuc->enabled =0 ; -/* Stop counting */ -amd_pmu_v2_disable_all (); +/* Stop counting but do not disable LBR */ +amd_pmu_core_disable_all (); status =amd_pmu_get_global_status ();
@@ -886,6 +895,12 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs)if (!status) goto done; +/* Read branch records before unfreezing */ +if (status &GLOBAL_STATUS_LBRS_FROZEN ) { +amd_pmu_lbr_read (); + status &= ~GLOBAL_STATUS_LBRS_FROZEN ; + } +for (idx =0 ; idx < x86_pmu.num_counters; idx++) {if (!test_bit (idx, cpuc->active_mask))continue ;
@@ -905,6 +920,9 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs)if (!x86_perf_event_set_period (event))continue ; +if (has_branch_stack (event)) + data.br_stack = &cpuc->lbr_stack; +if (perf_event_overflow (event, &data, regs))x86_pmu_stop (event,0 );
@@ -918,7 +936,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs)*/ WARN_ON (status >0 ); -/* Clear overflow bits */ +/* Clear overflow and freeze bits */ amd_pmu_ack_global_status (~status);/*
@@ -932,7 +950,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs)/* Resume counting only if PMU is active */ if (pmu_enabled) -amd_pmu_v2_enable_all (0 ); +amd_pmu_core_enable_all ();return amd_pmu_adjust_nmi_window (handled); }
@@ -1375,7 +1393,14 @@ static int __init amd_core_pmu_init(void) }/* LBR and BRS are mutually exclusive features */ -if (amd_pmu_lbr_init () && !amd_brs_init ()) { +if (!amd_pmu_lbr_init ()) { +/* LBR requires flushing on context switch */ + x86_pmu.sched_task = amd_pmu_lbr_sched_task; +static_call_update (amd_pmu_branch_hw_config, amd_pmu_lbr_hw_config); +static_call_update (amd_pmu_branch_reset, amd_pmu_lbr_reset); +static_call_update (amd_pmu_branch_add, amd_pmu_lbr_add); +static_call_update (amd_pmu_branch_del, amd_pmu_lbr_del); + }else if (!amd_brs_init ()) {/* * BRS requires special event constraints and flushing on ctxsw. */
arch/x86/events/amd/lbr.c +203 -0
209 lines not on this page · raw message
arch/x86/events/perf_event.h +8 -0
@@ -1231,6 +1231,14 @@ static inline bool fixed_counter_disabled(int i, struct pmu *pmu)int amd_pmu_init (void );int amd_pmu_lbr_init (void ); +void amd_pmu_lbr_reset (void ); +void amd_pmu_lbr_read (void ); +void amd_pmu_lbr_add (struct perf_event *event); +void amd_pmu_lbr_del (struct perf_event *event); +void amd_pmu_lbr_sched_task (struct perf_event_context *ctx,bool sched_in); +void amd_pmu_lbr_enable_all (void ); +void amd_pmu_lbr_disable_all (void ); +int amd_pmu_lbr_hw_config (struct perf_event *event);#ifdef CONFIG_PERF_EVENTS_AMD_BRS
arch/x86/include/asm/msr-index.h +5 -0
@@ -539,6 +539,9 @@#define MSR_AMD64_PERF_CNTR_GLOBAL_CTL 0xc0000301#define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR 0xc0000302 +/* AMD Last Branch Record MSRs */ +#define MSR_AMD64_LBR_SELECT 0xc000010e +/* Fam 17h MSRs */ #define MSR_F17H_IRPERF 0xc00000e9
@@ -707,6 +710,8 @@#define MSR_AMD_DBG_EXTN_CFG 0xc000010f#define MSR_AMD_SAMP_BR_FROM 0xc0010300 +#define DBG_EXTN_CFG_LBRV2EN BIT_ULL(6) +#define MSR_IA32_MPERF 0x000000e7#define MSR_IA32_APERF 0x000000e8
Signature · 1 line
2.34.1
PATCH 7/13 perf/x86/amd/lbr: Add LbrExtV2 hardware branch filter support
In reply to Sandipan DasIf AMD Last Branch Record Extension Version 2 (LbrExtV2) is detected, convert the requested branch filter (PERF_SAMPLE_BRANCH_* flags) to the corresponding hardware filter value and stash it in the event data when a branch stack is requested. The hardware filter value is also saved in per-CPU areas for use during event scheduling.
Hardware filtering is provided by the LBR Branch Select register. It has bits which when set, suppress recording of the following types of branches:
* CPL = 0 (Kernel only)
* CPL > 0 (Userspace only)
* Conditional Branches
* Near Relative Calls
* Near Indirect Calls
* Near Returns
* Near Indirect Jumps (excluding Near Indirect Calls and Near Returns)
* Near Relative Jumps (excluding Near Relative Calls)
* Far Branches
--- arch/x86/events/amd/core.c | 21 ++++++--- arch/x86/events/amd/lbr.c | 94 +++++++++++++++++++++++++++++++++++++- 2 files changed, 108 insertions(+), 7 deletions(-)
arch/x86/events/amd/core.c +15 -6
@@ -542,16 +542,24 @@ static int amd_pmu_cpu_prepare(int cpu) {struct cpu_hw_events *cpuc = &per_cpu (cpu_hw_events, cpu); + cpuc->lbr_sel =kzalloc_node (sizeof (struct er_account ),GFP_KERNEL , +cpu_to_node (cpu)); +if (!cpuc->lbr_sel) +return -ENOMEM ; +WARN_ON_ONCE (cpuc->amd_nb);if (!x86_pmu.amd_nb_constraints)return 0 ; cpuc->amd_nb =amd_alloc_nb (cpu); -if (!cpuc->amd_nb) -return -ENOMEM ; +if (cpuc->amd_nb) +return 0 ; -return 0 ; +kfree (cpuc->lbr_sel); + cpuc->lbr_sel =NULL ; + +return -ENOMEM ; }static void amd_pmu_cpu_starting (int cpu)
@@ -589,13 +597,14 @@ static void amd_pmu_cpu_starting(int cpu)static void amd_pmu_cpu_dead (int cpu) { -struct cpu_hw_events *cpuhw; +struct cpu_hw_events *cpuhw = &per_cpu (cpu_hw_events, cpu); + +kfree (cpuhw->lbr_sel); + cpuhw->lbr_sel =NULL ;if (!x86_pmu.amd_nb_constraints)return ; - cpuhw = &per_cpu (cpu_hw_events, cpu); -if (cpuhw->amd_nb) {struct amd_nb *nb = cpuhw->amd_nb;
arch/x86/events/amd/lbr.c +93 -1
140 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH 8/13 perf/x86: Move branch classifier
In reply to Sandipan DasCommit 3e702ff6d1ea ("perf/x86: Add LBR software filter support for Intel CPUs") introduces a software branch filter which complements the hardware branch filter and adds an x86 branch classifier.
Move the branch classifier to arch/x86/events/ so that it can be utilized by other vendors for branch record filtering.
--- arch/x86/events/Makefile | 2 +- arch/x86/events/intel/lbr.c | 273 ----------------------------------- arch/x86/events/perf_event.h | 62 ++++++++ arch/x86/events/utils.c | 216 +++++++++++++++++++++++++++ 4 files changed, 279 insertions(+), 274 deletions(-) create mode 100644 arch/x86/events/utils.c
arch/x86/events/Makefile +1 -1
@@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -obj-y += core.o probe.o +obj-y += core.o probe.o utils.o obj-$(CONFIG_PERF_EVENTS_INTEL_RAPL) += rapl.o obj-y += amd/ obj-$(CONFIG_X86_LOCAL_APIC) += msr.o
arch/x86/events/intel/lbr.c +0 -273
291 lines not on this page · raw message
arch/x86/events/perf_event.h +62 -0
68 lines not on this page · raw message
arch/x86/events/utils.c new file +216 -0
216 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH 9/13 perf/x86/amd/lbr: Add LbrExtV2 software branch filter support
In reply to Sandipan DasWith AMD Last Branch Record Extension Version 2 (LbrExtV2), it is necessary to process the branch records further as hardware filtering is not granular enough for identifying certain types of branches. E.g. to record system calls, one should record far branches. The filter captures both far calls and far returns but the irrelevant records are filtered out based on the branch type as seen by the branch classifier.
--- arch/x86/events/amd/lbr.c | 92 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 87 insertions(+), 5 deletions(-)
arch/x86/events/amd/lbr.c +87 -5
@@ -94,6 +94,50 @@ static __always_inline u64 sign_ext_branch_ip(u64 ip)return (u64)(((s64 )ip << shift) >> shift); } +static void amd_pmu_lbr_filter (void ) +{ +struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +int br_sel = cpuc->br_sel, type, i, j; +bool compress = false; +u64 from, to; + +/* If sampling all branches, there is nothing to filter */ +if (((br_sel &X86_BR_ALL ) ==X86_BR_ALL ) && + ((br_sel &X86_BR_TYPE_SAVE ) !=X86_BR_TYPE_SAVE )) +return ; + +for (i =0 ; i < cpuc->lbr_stack.nr; i++) { + from = cpuc->lbr_entries[i].from; + to = cpuc->lbr_entries[i].to; + type =branch_type (from, to,0 ); + +/* If type does not correspond, then discard */ +if (type ==X86_BR_NONE || (br_sel & type) != type) { + cpuc->lbr_entries[i].from =0 ;/* mark invalid */ + compress = true; + } + +if ((br_sel &X86_BR_TYPE_SAVE ) ==X86_BR_TYPE_SAVE ) + cpuc->lbr_entries[i].type =common_branch_type (type); + } + +if (!compress) +return ; + +/* Remove all invalid entries */ +for (i =0 ; i < cpuc->lbr_stack.nr; ) { +if (!cpuc->lbr_entries[i].from) { + j = i; +while (++j < cpuc->lbr_stack.nr) + cpuc->lbr_entries[j -1 ] = cpuc->lbr_entries[j]; + cpuc->lbr_stack.nr--; +if (!cpuc->lbr_entries[i].from) +continue ; + } + i++; + } +} +void amd_pmu_lbr_read (void ) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);
@@ -128,6 +172,9 @@ void amd_pmu_lbr_read(void)* LBR To[0] always represent the TOS */ cpuc->lbr_stack.hw_idx =0 ; + +/* Perform further software filtering */ +amd_pmu_lbr_filter (); }static const int lbr_select_map[PERF_SAMPLE_BRANCH_MAX_SHIFT ] = {
@@ -136,8 +183,8 @@ static const int lbr_select_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] = { [PERF_SAMPLE_BRANCH_HV_SHIFT] = LBR_IGNORE, [PERF_SAMPLE_BRANCH_ANY_SHIFT] = LBR_ANY, - [PERF_SAMPLE_BRANCH_ANY_CALL_SHIFT] = LBR_REL_CALL | LBR_IND_CALL, - [PERF_SAMPLE_BRANCH_ANY_RETURN_SHIFT] = LBR_RETURN, + [PERF_SAMPLE_BRANCH_ANY_CALL_SHIFT] = LBR_REL_CALL | LBR_IND_CALL | LBR_FAR, + [PERF_SAMPLE_BRANCH_ANY_RETURN_SHIFT] = LBR_RETURN | LBR_FAR, [PERF_SAMPLE_BRANCH_IND_CALL_SHIFT] = LBR_IND_CALL, [PERF_SAMPLE_BRANCH_ABORT_TX_SHIFT] = LBR_NOT_SUPP, [PERF_SAMPLE_BRANCH_IN_TX_SHIFT] = LBR_NOT_SUPP,
@@ -150,8 +197,6 @@ static const int lbr_select_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] = { [PERF_SAMPLE_BRANCH_NO_FLAGS_SHIFT] = LBR_NOT_SUPP, [PERF_SAMPLE_BRANCH_NO_CYCLES_SHIFT] = LBR_NOT_SUPP, - - [PERF_SAMPLE_BRANCH_TYPE_SAVE_SHIFT] = LBR_NOT_SUPP, };static int amd_pmu_lbr_setup_filter (struct perf_event *event)
@@ -165,6 +210,41 @@ static int amd_pmu_lbr_setup_filter(struct perf_event *event)if (!x86_pmu.lbr_nr)return -EOPNOTSUPP ; +if (br_type &PERF_SAMPLE_BRANCH_USER ) + mask |=X86_BR_USER ; + +if (br_type &PERF_SAMPLE_BRANCH_KERNEL ) + mask |=X86_BR_KERNEL ; + +/* Ignore BRANCH_HV here */ + +if (br_type &PERF_SAMPLE_BRANCH_ANY ) + mask |=X86_BR_ANY ; + +if (br_type &PERF_SAMPLE_BRANCH_ANY_CALL ) + mask |=X86_BR_ANY_CALL ; + +if (br_type &PERF_SAMPLE_BRANCH_ANY_RETURN ) + mask |=X86_BR_RET |X86_BR_IRET |X86_BR_SYSRET ; + +if (br_type &PERF_SAMPLE_BRANCH_IND_CALL ) + mask |=X86_BR_IND_CALL ; + +if (br_type &PERF_SAMPLE_BRANCH_COND ) + mask |=X86_BR_JCC ; + +if (br_type &PERF_SAMPLE_BRANCH_IND_JUMP ) + mask |=X86_BR_IND_JMP ; + +if (br_type &PERF_SAMPLE_BRANCH_CALL ) + mask |=X86_BR_CALL |X86_BR_ZERO_CALL ; + +if (br_type &PERF_SAMPLE_BRANCH_TYPE_SAVE ) + mask |=X86_BR_TYPE_SAVE ; + + reg->reg = mask; + mask =0 ; +for (i =0 ; i <PERF_SAMPLE_BRANCH_MAX_SHIFT ; i++) {if (!(br_type &BIT_ULL (i)))continue ;
@@ -220,13 +300,15 @@ void amd_pmu_lbr_reset(void)void amd_pmu_lbr_add (struct perf_event *event) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +struct hw_perf_event_extra *reg = &event->hw.branch_reg;if (!x86_pmu.lbr_nr)return ;if (has_branch_stack (event)) { cpuc->lbr_select =1 ; - cpuc->lbr_sel->config = event->hw.branch_reg.config; + cpuc->lbr_sel->config = reg->config; + cpuc->br_sel = reg->reg; }perf_sched_cb_inc (event->ctx->pmu);
Signature · 1 line
2.34.1
PATCH 10/13 perf/x86: Make branch classifier fusion-aware
In reply to Sandipan DasWith branch fusion and other optimizations, branch sampling hardware in some processors can report a branch from address that points to an instruction preceding the actual branch by several bytes.
In such cases, the classifier cannot determine the branch type which leads to failures such as with the recently added test from commit b55878c90ab9 ("perf test: Add test for branch stack sampling"). Branch information is also easier to consume and annotate if branch from addresses always point to branch instructions.
Add a new variant of the branch classifier that can account for instruction fusion. If fusion is expected and the current branch from address does not point to a branch instruction, it attempts to find the first branch within the next (MAX_INSN_SIZE - 1) bytes and if found, additionally provides the offset between the reported branch from address and the address of the expected branch instruction.
--- arch/x86/events/perf_event.h | 2 + arch/x86/events/utils.c | 169 +++++++++++++++++++++-------------- 2 files changed, 102 insertions(+), 69 deletions(-)
arch/x86/events/perf_event.h +2 -0
@@ -1269,6 +1269,8 @@ enum {int common_branch_type (int type);int branch_type (unsigned long from,unsigned long to,int abort); +int branch_type_fused (unsigned long from,unsigned long to,int abort, +int *offset);ssize_t x86_event_sysfs_show (char *page,u64 config,u64 event);ssize_t intel_event_sysfs_show (char *page,u64 config);
arch/x86/events/utils.c +100 -69
203 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH 11/13 perf/x86/amd/lbr: Use fusion-aware branch classifier
In reply to Sandipan DasAMD Last Branch Record Extension Version 2 (LbrExtV2) can report a branch from address that points to an instruction preceding the actual branch by several bytes due to branch fusion and further optimizations in Zen4 processors.
In such cases, software should move forward sequentially in the instruction stream from the reported address and the address of the first branch encountered should be used instead. Hence, use the fusion-aware branch classifier to determine the correct branch type and get the offset for adjusting the branch from address.
--- arch/x86/events/amd/lbr.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/events/amd/lbr.c +10 -2
@@ -97,7 +97,7 @@ static __always_inline u64 sign_ext_branch_ip(u64 ip)static void amd_pmu_lbr_filter (void ) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); -int br_sel = cpuc->br_sel, type, i, j; +int br_sel = cpuc->br_sel, offset, type, i, j;bool compress = false;u64 from, to;
@@ -109,7 +109,15 @@ static void amd_pmu_lbr_filter(void)for (i =0 ; i < cpuc->lbr_stack.nr; i++) { from = cpuc->lbr_entries[i].from; to = cpuc->lbr_entries[i].to; - type =branch_type (from, to,0 ); + type =branch_type_fused (from, to,0 , &offset); + +/* +* Adjust the branch from address in case of instruction +* fusion where it points to an instruction preceding the +* actual branch +*/ +if (offset) + cpuc->lbr_entries[i].from += offset;/* If type does not correspond, then discard */ if (type ==X86_BR_NONE || (br_sel & type) != type) {
Signature · 1 line
2.34.1
PATCH 12/13 perf/core: Add speculation info to branch entries
In reply to Sandipan DasAdd a new "spec" bitfield to branch entries for providing speculation information. This will be populated using hints provided by branch sampling features on supported hardware. The following cases are covered:
* No branch speculation information is available
* Branch is speculative but taken on the wrong path
* Branch is non-speculative but taken on the correct path
* Branch is speculative and taken on the correct path
--- include/linux/perf_event.h | 1 + include/uapi/linux/perf_event.h | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)
include/linux/perf_event.h +1 -0
@@ -1078,6 +1078,7 @@ static inline void perf_clear_branch_entry_bitfields(struct perf_branch_entry *b br->abort =0 ; br->cycles =0 ; br->type =0 ; + br->spec =PERF_BR_SPEC_NA ; br->reserved =0 ; }
include/uapi/linux/perf_event.h +14 -1
@@ -256,6 +256,17 @@ enum { PERF_BR_MAX, }; +/* +* Common branch speculation outcome classification +*/ +enum { +PERF_BR_SPEC_NA =0 ,/* Not available */ +PERF_BR_SPEC_WRONG_PATH =1 ,/* Speculative but on wrong path */ +PERF_BR_NON_SPEC_CORRECT_PATH =2 ,/* Non-speculative but on correct path */ +PERF_BR_SPEC_CORRECT_PATH =3 ,/* Speculative and on correct path */ +PERF_BR_SPEC_MAX , +}; +#define PERF_SAMPLE_BRANCH_PLM_ALL \ (PERF_SAMPLE_BRANCH_USER|\ PERF_SAMPLE_BRANCH_KERNEL|\
@@ -1363,6 +1374,7 @@ union perf_mem_data_src {* abort: aborting a hardware transaction * cycles: cycles from last branch (or 0 if not supported) * type: branch type +* spec: branch speculation info (or 0 if not supported) */ struct perf_branch_entry {__u64 from;
@@ -1373,7 +1385,8 @@ struct perf_branch_entry { abort:1, /* transaction abort */ cycles:16, /* cycle count to last branch */ type:4, /* branch type */ - reserved:40; + spec:2, /* branch speculation info */ + reserved:38; };union perf_sample_weight {
Signature · 1 line
2.34.1
PATCH 13/13 perf/x86/amd/lbr: Add LbrExtV2 branch speculation info support
In reply to Sandipan DasProvide branch speculation information captured via AMD Last Branch Record Extension Version 2 (LbrExtV2) by setting the speculation info in branch records. The info is based on the "valid" and "spec" bits in the Branch To registers.
--- arch/x86/events/amd/lbr.c | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-)
arch/x86/events/amd/lbr.c +32 -3
@@ -146,12 +146,19 @@ static void amd_pmu_lbr_filter(void) } } +static const int lbr_spec_map[PERF_BR_SPEC_MAX ] = { +PERF_BR_SPEC_NA , +PERF_BR_SPEC_WRONG_PATH , +PERF_BR_NON_SPEC_CORRECT_PATH , +PERF_BR_SPEC_CORRECT_PATH , +}; +void amd_pmu_lbr_read (void ) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);struct perf_branch_entry *br = cpuc->lbr_entries;struct branch_entry entry; -int out =0 , i; +int out =0 , idx, i;if (!cpuc->lbr_users)return ;
@@ -160,8 +167,11 @@ void amd_pmu_lbr_read(void) entry.from.full =amd_pmu_lbr_get_from (i); entry.to.full =amd_pmu_lbr_get_to (i); -/* Check if a branch has been logged */ -if (!entry.to.split.valid) +/* +* Check if a branch has been logged; if valid = 0, spec = 0 +* then no branch was recorded +*/ +if (!entry.to.split.valid && !entry.to.split.spec)continue ;perf_clear_branch_entry_bitfields (br + out);
@@ -170,6 +180,25 @@ void amd_pmu_lbr_read(void) br[out].to =sign_ext_branch_ip (entry.to.split.ip); br[out].mispred = entry.from.split.mispredict; br[out].predicted = !br[out].mispred; + +/* +* Set branch speculation information using the status of +* the valid and spec bits. +* +* When valid = 0, spec = 0, no branch was recorded and the +* entry is discarded as seen above. +* +* When valid = 0, spec = 1, the recorded branch was +* speculative but took the wrong path. +* +* When valid = 1, spec = 0, the recorded branch was +* non-speculative but took the correct path. +* +* When valid = 1, spec = 1, the recorded branch was +* speculative and took the correct path +*/ + idx = (entry.to.split.valid <<1 ) | entry.to.split.spec; + br[out].spec = lbr_spec_map[idx]; out++; }
Signature · 1 line
2.34.1
Re: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit
In reply to Sandipan DasQuoted 9 lines · Sandipan Das
CPUID leaf
0x80000022i.e. ExtPerfMonAndDbg advertises some new performance monitoring features for AMD processors.Bit 1 of EAX indicates support for Last Branch Record Extension Version 2 (LbrExtV2) features. If found to be set during PMU initialization, the EBX bits of the same leaf can be used to determine the number of available LBR entries.
For better utilization of feature words, LbrExtV2 is added as a scattered
s/LbrExtV2 is added/add LbrExtV2/
Quoted 26 lines
feature bit.
Signed-off-by Sandipan Das <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +1
-1@@ -96,7 +96,7 @@#define X86_FEATURE_SYSCALL32 ( 3*32+14)/* "" syscall in IA32 userspace */ #define X86_FEATURE_SYSENTER32 ( 3*32+15)/* "" sysenter in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16)/* REP microcode works well */ -/* FREE! ( 3*32+17) */ +#define X86_FEATURE_LBREXT_V2 ( 3*32+17)/* AMD Last Branch Record Extension Version 2 */ #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18)/* "" LFENCE synchronizes RDTSC */ #define X86_FEATURE_ACC_POWER ( 3*32+19)/* AMD Accumulated Power Mechanism */ #define X86_FEATURE_NOPL ( 3*32+20)/* The NOPL (0F 1F) instructions */ arch/x86/kernel/cpu/scattered.c +0
-0
arch/x86/kernel/cpu/scattered.c static const struct cpuid_bit cpuid_bits[] = { · line 49
Quoted 6 lines
arch/x86/kernel/cpu/scattered.c +1
-0@@ -44,6 +44,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, { X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 }, + { X86_FEATURE_LBREXT_V2, CPUID_EAX, 1, 0x80000022, 0 }, { 0, 0, 0, 0, 0 } };
Other than that:
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquettePeter Zijlstra
maintainerRe: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit
In reply to Sandipan DasQuoted 35 lines · Sandipan Das
CPUID leaf
0x80000022i.e. ExtPerfMonAndDbg advertises some new performance monitoring features for AMD processors.Bit 1 of EAX indicates support for Last Branch Record Extension Version 2 (LbrExtV2) features. If found to be set during PMU initialization, the EBX bits of the same leaf can be used to determine the number of available LBR entries.
For better utilization of feature words, LbrExtV2 is added as a scattered feature bit.
Signed-off-by Sandipan Das <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +1
-1@@ -96,7 +96,7 @@#define X86_FEATURE_SYSCALL32 ( 3*32+14)/* "" syscall in IA32 userspace */ #define X86_FEATURE_SYSENTER32 ( 3*32+15)/* "" sysenter in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16)/* REP microcode works well */ -/* FREE! ( 3*32+17) */ +#define X86_FEATURE_LBREXT_V2 ( 3*32+17)/* AMD Last Branch Record Extension Version 2 */ #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18)/* "" LFENCE synchronizes RDTSC */ #define X86_FEATURE_ACC_POWER ( 3*32+19)/* AMD Accumulated Power Mechanism */ #define X86_FEATURE_NOPL ( 3*32+20)/* The NOPL (0F 1F) instructions */ arch/x86/kernel/cpu/scattered.c +0
-0
arch/x86/kernel/cpu/scattered.c static const struct cpuid_bit cpuid_bits[] = { · line 49
Quoted 6 lines
arch/x86/kernel/cpu/scattered.c +1
-0@@ -44,6 +44,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, { X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 }, + { X86_FEATURE_LBREXT_V2, CPUID_EAX, 1, 0x80000022, 0 }, { 0, 0, 0, 0, 0 } };
Would LBR_V2 work at all? It being a new version already seems to imply extention, no? Then again, I suppose there's an argument to be had for avoiding confusion vs the Intel LBR thing.. Couldn't you have called this BRS_V2 :-)
Oh well...
Re: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit
Hi,
Quoted 40 lines · Peter Zijlstra
Quoted 38 lines · Sandipan Das
CPUID leaf
0x80000022i.e. ExtPerfMonAndDbg advertises some new performance monitoring features for AMD processors.Bit 1 of EAX indicates support for Last Branch Record Extension Version 2 (LbrExtV2) features. If found to be set during PMU initialization, the EBX bits of the same leaf can be used to determine the number of available LBR entries.
For better utilization of feature words, LbrExtV2 is added as a scattered feature bit.
Signed-off-by Sandipan Das <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +1
-1@@ -96,7 +96,7 @@#define X86_FEATURE_SYSCALL32 ( 3*32+14)/* "" syscall in IA32 userspace */ #define X86_FEATURE_SYSENTER32 ( 3*32+15)/* "" sysenter in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16)/* REP microcode works well */ -/* FREE! ( 3*32+17) */ +#define X86_FEATURE_LBREXT_V2 ( 3*32+17)/* AMD Last Branch Record Extension Version 2 */ #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18)/* "" LFENCE synchronizes RDTSC */ #define X86_FEATURE_ACC_POWER ( 3*32+19)/* AMD Accumulated Power Mechanism */ #define X86_FEATURE_NOPL ( 3*32+20)/* The NOPL (0F 1F) instructions */ arch/x86/kernel/cpu/scattered.c +0
-0@@ -44,6 +44,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, { X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 },
Quoted 9 lines
arch/x86/kernel/cpu/scattered.c static const struct cpuid_bit cpuid_bits[] = { · line 49
Quoted 3 lines
arch/x86/kernel/cpu/scattered.c +1
-0+ { X86_FEATURE_LBREXT_V2, CPUID_EAX, 1, 0x80000022, 0 }, { 0, 0, 0, 0, 0 } };Would
LBR_V2work at all? It being a new version already seems to imply extention, no? Then again, I suppose there's an argument to be had for avoiding confusion vs the Intel LBR thing.. Couldn't you have called thisBRS_V2:-)
I believe it is called v2 because there was already a LBR in previous generations, however it
was 1-deep and it was not connected to the PMU like this one. The public PPR mentions it
(MSR 0x1DB/0x1DC, Last Branch From IP, Last Branch To IP). See for instance the PPR
for Fam17h model 71h:
https://www.amd.com/system/files/TechDocs/56176_ppr_Family_17h_Model_71h_B0_pub_Rev_3.06.zip
BRS is a model specific feature for Zen3.
LBRv2 is a great improvement including over Zen3 BRS.
Peter Zijlstra
maintainerRe: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit
Quoted 52 lines · Stephane Eranian
Hi,
Quoted 40 lines · Peter Zijlstra
Quoted 38 lines · Sandipan Das
CPUID leaf
0x80000022i.e. ExtPerfMonAndDbg advertises some new performance monitoring features for AMD processors.Bit 1 of EAX indicates support for Last Branch Record Extension Version 2 (LbrExtV2) features. If found to be set during PMU initialization, the EBX bits of the same leaf can be used to determine the number of available LBR entries.
For better utilization of feature words, LbrExtV2 is added as a scattered feature bit.
Signed-off-by Sandipan Das <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +1
-1@@ -96,7 +96,7 @@#define X86_FEATURE_SYSCALL32 ( 3*32+14)/* "" syscall in IA32 userspace */ #define X86_FEATURE_SYSENTER32 ( 3*32+15)/* "" sysenter in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16)/* REP microcode works well */ -/* FREE! ( 3*32+17) */ +#define X86_FEATURE_LBREXT_V2 ( 3*32+17)/* AMD Last Branch Record Extension Version 2 */ #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18)/* "" LFENCE synchronizes RDTSC */ #define X86_FEATURE_ACC_POWER ( 3*32+19)/* AMD Accumulated Power Mechanism */ #define X86_FEATURE_NOPL ( 3*32+20)/* The NOPL (0F 1F) instructions */ arch/x86/kernel/cpu/scattered.c +0
-0@@ -44,6 +44,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, { X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 },Quoted 9 lines
arch/x86/kernel/cpu/scattered.c static const struct cpuid_bit cpuid_bits[] = { · line 49
Quoted 3 lines
arch/x86/kernel/cpu/scattered.c +1
-0+ { X86_FEATURE_LBREXT_V2, CPUID_EAX, 1, 0x80000022, 0 }, { 0, 0, 0, 0, 0 } };Would
LBR_V2work at all? It being a new version already seems to imply extention, no? Then again, I suppose there's an argument to be had for avoiding confusion vs the Intel LBR thing.. Couldn't you have called thisBRS_V2:-)
Quoted 2 lines
I believe it is called v2 because there was already a LBR in previous generations, however it
That's not the question; It's currently called LBREXT_V2, which is a bit of a shit name. Then again LBR_V2 is too because AMD and Intel LBR are quite different. So in that respect BRS_V2 would be an ever so much better name.
Re: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit
Hi Peter,
Quoted 53 lines · Peter Zijlstra
Quoted 52 lines · Stephane Eranian
Hi,
Quoted 49 lines · Peter Zijlstra
Quoted 35 lines · Sandipan Das
CPUID leaf
0x80000022i.e. ExtPerfMonAndDbg advertises some new performance monitoring features for AMD processors.Bit 1 of EAX indicates support for Last Branch Record Extension Version 2 (LbrExtV2) features. If found to be set during PMU initialization, the EBX bits of the same leaf can be used to determine the number of available LBR entries.
For better utilization of feature words, LbrExtV2 is added as a scattered feature bit.
Signed-off-by Sandipan Das <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +1
-1@@ -96,7 +96,7 @@#define X86_FEATURE_SYSCALL32 ( 3*32+14)/* "" syscall in IA32 userspace */ #define X86_FEATURE_SYSENTER32 ( 3*32+15)/* "" sysenter in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16)/* REP microcode works well */ -/* FREE! ( 3*32+17) */ +#define X86_FEATURE_LBREXT_V2 ( 3*32+17)/* AMD Last Branch Record Extension Version 2 */ #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18)/* "" LFENCE synchronizes RDTSC */ #define X86_FEATURE_ACC_POWER ( 3*32+19)/* AMD Accumulated Power Mechanism */ #define X86_FEATURE_NOPL ( 3*32+20)/* The NOPL (0F 1F) instructions */ arch/x86/kernel/cpu/scattered.c +0
-0arch/x86/kernel/cpu/scattered.c static const struct cpuid_bit cpuid_bits[] = { · line 49
Quoted 6 lines
arch/x86/kernel/cpu/scattered.c +1
-0@@ -44,6 +44,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, { X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 }, + { X86_FEATURE_LBREXT_V2, CPUID_EAX, 1, 0x80000022, 0 }, { 0, 0, 0, 0, 0 } };Would
LBR_V2work at all? It being a new version already seems to imply extention, no? Then again, I suppose there's an argument to be had for avoiding confusion vs the Intel LBR thing.. Couldn't you have called thisBRS_V2:-)
Quoted 7 lines
Quoted 2 lines
I believe it is called v2 because there was already a LBR in previous generations, however it
That's not the question; It's currently called
LBREXT_V2, which is a bit of a shit name. Then againLBR_V2is too because AMD and Intel LBR are quite different. So in that respectBRS_V2would be an ever so much better name.
AMD LbrExtV2 is similar to Intel LBR. Unlike BRS, LbrExtV2 does not rely on interrupt holding. The branch records are "frozen" at the time of counter overflow.
- Sandipan
Peter Zijlstra
maintainerRe: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit
Quoted 1 line · Sandipan Das
AMD LbrExtV2 is similar to Intel LBR. Unlike BRS, LbrExtV2 does not rely on
LbrExtV2 must be the most terrible name ever, please stop using it. Heck your own code calls it lbr_v2 wherever it can.
So can we please just kill that name entirely?
$ quilt diff --combine - | grep -i lbrext_v2
+ if (x86_pmu.version < 2 || !boot_cpu_has(X86_FEATURE_LBREXT_V2))
+#define X86_FEATURE_LBREXT_V2 ( 3*32+17) /* AMD Last Branch Record Extension Version 2 */
+ { X86_FEATURE_LBREXT_V2, CPUID_EAX, 1, 0x80000022, 0 },Is the complete usage of this silly name.
Quoted 2 lines
interrupt holding. The branch records are "frozen" at the time of counter overflow.
Yes, I get all that. It is also significantly different from Intel LBR in all details and shares not a single line of code, so also calling it LBR is confusing at best.
The MSRs are called AMD_SAMPL_BR, so why not call the thing BRS_V2 ?
Re: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit
Hi Peter,
Quoted 13 lines · Peter Zijlstra
Quoted 1 line · Sandipan Das
AMD LbrExtV2 is similar to Intel LBR. Unlike BRS, LbrExtV2 does not rely on
LbrExtV2 must be the most terrible name ever, please stop using it. Heck your own code calls it
lbr_v2wherever it can.So can we please just kill that name entirely?
$ quilt diff --combine - | grep -i lbrext_v2(quoted diff) +3
-0+ if (x86_pmu.version < 2 || !boot_cpu_has(X86_FEATURE_LBREXT_V2)) +#define X86_FEATURE_LBREXT_V2 ( 3*32+17) /* AMD Last Branch Record Extension Version 2 */ + { X86_FEATURE_LBREXT_V2, CPUID_EAX, 1, 0x80000022, 0 },
Quoted 2 lines
Is the complete usage of this silly name.
I don't have any reservations about the name :) Its the name of the feature bit from CPUID 0x80000022 EAX as mentioned in the processor manuals.
Unlike BRS, LBRv2 (if I may call it so) is an architected feature starting with Zen4. BRS is model-specific and available only on Zen3. LBRv2 supersedes it and from an architectural perspective, future platforms are slated to have LBRv2 and not BRS. As Stephane alluded to earlier on this thread, AMD's legacy LBR is 1-entry deep. We want to be able to differentiate from that as well as BRS. For these reasons, and also to keep it disjoint from Intel's LBR, if you so prefer, I can rename this feature to AMD_LBR_V2.
Quoted 8 lines
Quoted 2 lines
interrupt holding. The branch records are "frozen" at the time of counter overflow.
Yes, I get all that. It is also significantly different from Intel LBR in all details and shares not a single line of code, so also calling it LBR is confusing at best.
The MSRs are called
AMD_SAMPL_BR, so why not call the thingBRS_V2?
Sorry for the confusion with the register names. Since the SAMP_BR_FROM and SAMP_BR_TO registers used by BRS have the same addresses as that of the LBR_TO_V2 and LBR_FROM_V2 registers, I chose to reuse the definitions.
It it helps and considering LBRv2 is an architected feature, I can rename AMD_SAMP_BR_* to AMD_LBR_V2_* or have these MSR definitions duplicated with different names.
- Sandipan
Peter Zijlstra
maintainerRe: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit
Quoted 1 line · Sandipan Das
I can rename this feature to
AMD_LBR_V2.
I've done that and stuck them in queue/perf/core. If the robots don't hate on it I'll push them into tip.
Re: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Sandipan Das
I can rename this feature to
AMD_LBR_V2.I've done that and stuck them in queue/perf/core. If the robots don't hate on it I'll push them into tip.
Thanks!