Skip to contentSkip to conversation

perf/x86/amd: Add AMD Fam19h Branch Sampling support

PATCH Applied Stephane Eranian 5 participants 34 messages Mar 22, 2022 – Apr 5, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v7 cover perf/x86/amd: Add AMD Fam19h Branch Sampling support

Add support for the AMD Fam19h 16-deep branch sampling feature as described in the AMD PPR Fam19h Model 01h Revision B1 section 2.1.13. This is a model specific extension. It is not an architected AMD feature.

The Branch Sampling Feature (BRS) provides the statistical taken branch information necessary to enable autoFDO-style optimization by compilers, i.e., basic block execution counts.

BRS operates with a 16-deep saturating buffer in MSR registers. There is no hardware branch type filtering. All control flow changes are captured. BRS relies on specific programming of the core PMU of Fam19h. In particular, the following requirements must be met:
- the sampling period be greater than 16 (BRS depth)
- the sampling period must use fixed and not frequency mode

BRS interacts with the NMI interrupt as well. Because enabling BRS is expensive, it is only activated after P event occurrences, where P is the desired sampling period. At P occurrences of the event, the counter overflows, the CPU catches the NMI interrupt, activates BRS for 16 branches until it saturates, and then delivers the NMI to the kernel. Between the overflow and the time BRS activates more branches may be executed skewing the period. All along, the sampling event keeps counting. The skid may be attenuated by reducing the sampling period by 16.

BRS is integrated into perf_events seamlessly via the same PERF_RECORD_BRANCH_STACK sample format. BRS generates branch perf_branch_entry records in the sampling buffer. There is no prediction or latency information supported. The branches are stored in reverse order of execution. The most recent branch is the first entry in each record.

Because BRS must be stopped when a CPU goes into low power mode, the series includes patches to add callbacks on ACPI low power entry and exit which is used on AMD processors.

Given that there is no privilege filterting with BRS, the kernel implements filtering on privlege level.

This version adds a few simple modifications to perf record and report.
1. add the branch-brs event as a builtin such as it can used directly:
perf record -e branch-brs ...
2. improve error handling for AMD IBS and is contributed by Kim Phillips.
3. use the better error handling to improve error handling for BRS.
4. add two new sort dimensions to help display the branch sampling information. Because there is no latency information associated with the branch sampling feature perf report would collapse all samples within a function into a single histogram entry. This is expected because the default sort mode for PERF_SAMPLE_BRANCH_STACK is symbol_from/symbol_to. This propagates to the annotation.

For more detailed view of the branch samples, the new sort dimensions addr_from,addr_to can be used instead as follows:

$ perf report --sort=overhead,comm,dso,addr_from,addr_to 
# Overhead  Command    Shared Object     Source Address   Target Address
# ........  .......... ..............  ..............     ..............
#
     4.21%  test_prg   test_prg       [.] test_threa+0x3c [.] test_threa+0x4
     4.14%  test_prg   test_prg       [.] test_threa+0x3e [.] test_threa+0x2
     4.10%  test_prg   test_prg       [.] test_threa+0x4  [.] test_threa+0x3a
     4.07%  test_prg   test_prg       [.] test_threa+0x2  [.] test_threa+0x3c

Versus the default output:

$ perf report 
# Overhead  Command   Source Shared Object Source Symbol    Target Symbol      Basic Block Cycles
# ........  ......... .................... ................ .................  ..................
#
    99.52%  test_prg  test_prg             [.] test_thread  [.] test_thread    -                 

BRS can be used with any sampling event. However, it is recommended to use the RETIRED_BRANCH event because it matches what the BRS captures. For convenience, a pseudo event matching the branches captured by BRS is exported by the kernel (branch-brs):

$ perf record -b -e cpu/branch-brs/ -c 1000037 test
$ perf report -D
56531696056126 0x193c000 [0x1a8]: PERF_RECORD_SAMPLE(IP, 0x2): 18122/18230: 0x401d24 period: 1000037 addr: 0
... branch stack: nr:16
.....  0: 0000000000401d24 -> 0000000000401d5a 0 cycles      0
.....  1: 0000000000401d5c -> 0000000000401d24 0 cycles      0
.....  2: 0000000000401d22 -> 0000000000401d5c 0 cycles      0
.....  3: 0000000000401d5e -> 0000000000401d22 0 cycles      0
.....  4: 0000000000401d20 -> 0000000000401d5e 0 cycles      0
.....  5: 0000000000401d3e -> 0000000000401d20 0 cycles      0
.....  6: 0000000000401d42 -> 0000000000401d3e 0 cycles      0
.....  7: 0000000000401d3c -> 0000000000401d42 0 cycles      0
.....  8: 0000000000401d44 -> 0000000000401d3c 0 cycles      0
.....  9: 0000000000401d3a -> 0000000000401d44 0 cycles      0
..... 10: 0000000000401d46 -> 0000000000401d3a 0 cycles      0
..... 11: 0000000000401d38 -> 0000000000401d46 0 cycles      0
..... 12: 0000000000401d48 -> 0000000000401d38 0 cycles      0
..... 13: 0000000000401d36 -> 0000000000401d48 0 cycles      0
..... 14: 0000000000401d4a -> 0000000000401d36 0 cycles      0
..... 15: 0000000000401d34 -> 0000000000401d4a 0 cycles      0
 ... thread: test:18230
 ...... dso: test

Special thanks to Kim Phillips @ AMD for the testing, reviews and contributions.

V2 makes the following changes:
- the low power callback code has be reworked completly. It is not impacting the generic perf_events code anymore. This is all handled via x86 code and only for ACPI low power driver which seems to be the default on AMD. The change in acpi_pad.c and processor_idle.c has no impact on non x86 architectures, on Intel x86 or AMD without BRS, a jump label is used to void the code unless necessary
- BRS is an opt-in compile time option for the kernel
- branch_stack bit clearing helper is introduced
- As for the fact that BRS holds the NMI and that it may conflict with other sampling events and introduced skid, this is not really a problem because AMD PMI skid is already very large prompting special handling in amd_pmu_wait_on_overflow(), so adding a few cycles while the CPU executes at most 16 taken branches is not a problem.

V3 makes the following changes:
- simplifies the handling of BRS enable/disable to mimic the Intel LBR code path more closely. That removes some callbacks in generic x86 code
- add config option to compile BRS as an opt-in (off by default)
- updated perf tool error reporting patch updates by Kim Phillips

V4 makes the following changes:
- rebase to latest tip.git (commit 6f5ac142e5df)
- integrate Kim Phillips latest perf tool error handling patches

V5 makes the following changes:
- rebased to 5.17-rc1
- integrated feedback from PeterZ about AMD perf_events callbacks
- fix cpufeature macros name X86_FEATURE_BRS
- integrated all perf tool error handling from kim Phillips

V6 makes the following changes:
- rebase to 5.17-rc3
- fix the typo in the Kconfig for BRS opt-in
- reword all changelogs to be more aligned with standard

V7 makes the following changes:
- rebased to 5.17-rc8
- Restrict BRS to be used only with RETIRED_TAKEN_BRANCH event in order to avoid adding more skid to other events in case that are monitored concurrently
- fixed bugs in the perf tool enhanced error handling for IBS and BRS

Kim Phillips (1):
perf tools: Improve IBS error handling

Stephane Eranian (12):
perf/core: add perf_clear_branch_entry_bitfields() helper x86/cpufeatures: add AMD Fam19h Branch Sampling feature perf/x86/amd: add AMD Fam19h Branch Sampling support perf/x86/amd: add branch-brs helper event for Fam19h BRS perf/x86/amd: enable branch sampling priv level filtering perf/x86/amd: add AMD branch sampling period adjustment perf/x86/amd: make Zen3 branch sampling opt-in
ACPI: add perf low power callback
perf/x86/amd: add idle hooks for branch sampling perf tools: fix NULL point in evsel__env()
perf tools: Improve error handling of AMD Branch Sampling perf report: add addr_from/addr_to sort dimensions

 arch/x86/events/Kconfig            |   8 +
 arch/x86/events/amd/Makefile       |   1 +
 arch/x86/events/amd/brs.c          | 363 +++++++++++++++++++++++++++++
 arch/x86/events/amd/core.c         | 252 +++++++++++++++++++-
 arch/x86/events/core.c             |  17 +-
 arch/x86/events/intel/lbr.c        |  36 ++-
 arch/x86/events/perf_event.h       | 143 ++++++++++--
 arch/x86/include/asm/cpufeatures.h |   1 +
 arch/x86/include/asm/msr-index.h   |   4 +
 arch/x86/include/asm/perf_event.h  |  21 ++
 drivers/acpi/acpi_pad.c            |   6 +
 drivers/acpi/processor_idle.c      |   5 +
 include/linux/perf_event.h         |  22 ++
 tools/perf/util/evsel.c            |  48 +++-
 tools/perf/util/hist.c             |   2 +
 tools/perf/util/hist.h             |   2 +
 tools/perf/util/sort.c             | 128 ++++++++++
 tools/perf/util/sort.h             |   2 +
 18 files changed, 1020 insertions(+), 41 deletions(-)
 create mode 100644 arch/x86/events/amd/brs.c
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#1 lore raw

PATCH v7 1/13 perf/core: add perf_clear_branch_entry_bitfields() helper

Make it simpler to reset all the info fields on the perf_branch_entry by adding a helper inline function.

The goal is to centralize the initialization to avoid missing a field in case more are added.

---
 arch/x86/events/intel/lbr.c | 36 +++++++++++++++++-------------------
 include/linux/perf_event.h  | 16 ++++++++++++++++
 2 files changed, 33 insertions(+), 19 deletions(-)
arch/x86/events/intel/lbr.c +17 -19
index fe1742c4ca49..13179f31fe10 100644
@@ -769,6 +769,7 @@ void intel_pmu_lbr_disable_all(void)
 void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
 {
 	unsigned long mask = x86_pmu.lbr_nr - 1;
+	struct perf_branch_entry *br = cpuc->lbr_entries;
 	u64 tos = intel_pmu_lbr_tos();
 	int i;
 
@@ -784,15 +785,11 @@ void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
 
 		rdmsrl(x86_pmu.lbr_from + lbr_idx, msr_lastbranch.lbr);
 
-		cpuc->lbr_entries[i].from	= msr_lastbranch.from;
-		cpuc->lbr_entries[i].to		= msr_lastbranch.to;
-		cpuc->lbr_entries[i].mispred	= 0;
-		cpuc->lbr_entries[i].predicted	= 0;
-		cpuc->lbr_entries[i].in_tx	= 0;
-		cpuc->lbr_entries[i].abort	= 0;
-		cpuc->lbr_entries[i].cycles	= 0;
-		cpuc->lbr_entries[i].type	= 0;
-		cpuc->lbr_entries[i].reserved	= 0;
+		perf_clear_branch_entry_bitfields(br);
+
+		br->from	= msr_lastbranch.from;
+		br->to		= msr_lastbranch.to;
+		br++;
 	}
 	cpuc->lbr_stack.nr = i;
 	cpuc->lbr_stack.hw_idx = tos;
@@ -807,6 +804,7 @@ void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)
 {
 	bool need_info = false, call_stack = false;
 	unsigned long mask = x86_pmu.lbr_nr - 1;
+	struct perf_branch_entry *br = cpuc->lbr_entries;
 	u64 tos = intel_pmu_lbr_tos();
 	int i;
 	int out = 0;
@@ -878,15 +876,14 @@ void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)
 		if (abort && x86_pmu.lbr_double_abort && out > 0)
 			out--;
 
-		cpuc->lbr_entries[out].from	 = from;
-		cpuc->lbr_entries[out].to	 = to;
-		cpuc->lbr_entries[out].mispred	 = mis;
-		cpuc->lbr_entries[out].predicted = pred;
-		cpuc->lbr_entries[out].in_tx	 = in_tx;
-		cpuc->lbr_entries[out].abort	 = abort;
-		cpuc->lbr_entries[out].cycles	 = cycles;
-		cpuc->lbr_entries[out].type	 = 0;
-		cpuc->lbr_entries[out].reserved	 = 0;
+		perf_clear_branch_entry_bitfields(br+out);
+		br[out].from	 = from;
+		br[out].to	 = to;
+		br[out].mispred	 = mis;
+		br[out].predicted = pred;
+		br[out].in_tx	 = in_tx;
+		br[out].abort	 = abort;
+		br[out].cycles	 = cycles;
 		out++;
 	}
 	cpuc->lbr_stack.nr = out;
@@ -951,6 +948,8 @@ static void intel_pmu_store_lbr(struct cpu_hw_events *cpuc,
 		to = rdlbr_to(i, lbr);
 		info = rdlbr_info(i, lbr);
 
+		perf_clear_branch_entry_bitfields(e);
+
 		e->from		= from;
 		e->to		= to;
 		e->mispred	= get_lbr_mispred(info);
@@ -959,7 +958,6 @@ static void intel_pmu_store_lbr(struct cpu_hw_events *cpuc,
 		e->abort	= !!(info & LBR_INFO_ABORT);
 		e->cycles	= get_lbr_cycles(info);
 		e->type		= get_lbr_br_type(info);
-		e->reserved	= 0;
 	}
 
 	cpuc->lbr_stack.nr = i;
include/linux/perf_event.h +16 -0
index 733649184b27..496eb6aa6e54 100644
@@ -1063,6 +1063,22 @@ static inline void perf_sample_data_init(struct perf_sample_data *data,
 	data->txn = 0;
 }
 
+/*
+ * Clear all bitfields in the perf_branch_entry.
+ * The to and from fields are not cleared because they are
+ * systematically modified by caller.
+ */
+static inline void perf_clear_branch_entry_bitfields(struct perf_branch_entry *br)
+{
+	br->mispred = 0;
+	br->predicted = 0;
+	br->in_tx = 0;
+	br->abort = 0;
+	br->cycles = 0;
+	br->type = 0;
+	br->reserved = 0;
+}
+
 extern void perf_output_sample(struct perf_output_handle *handle,
 			       struct perf_event_header *header,
 			       struct perf_sample_data *data,
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#2 lore raw

[tip: perf/core] perf/core: Add perf_clear_branch_entry_bitfields() helper

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     bfe4daf850f45d92dcd3da477f0b0456620294c3
Gitweb:        https://git.kernel.org/tip/bfe4daf850f45d92dcd3da477f0b0456620294c3
Author:        Stephane Eranian <[email protected]>
AuthorDate:    Tue, 22 Mar 2022 15:15:05 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 05 Apr 2022 10:24:36 +02:00

perf/core: Add perf_clear_branch_entry_bitfields() helper

Make it simpler to reset all the info fields on the perf_branch_entry by adding a helper inline function.

The goal is to centralize the initialization to avoid missing a field in case more are added.

---
 arch/x86/events/intel/lbr.c | 36 +++++++++++++++++-------------------
 include/linux/perf_event.h  | 16 ++++++++++++++++-
 2 files changed, 33 insertions(+), 19 deletions(-)
arch/x86/events/intel/lbr.c +17 -19
index fe1742c..13179f3 100644
@@ -769,6 +769,7 @@ void intel_pmu_lbr_disable_all(void)
 void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
 {
 	unsigned long mask = x86_pmu.lbr_nr - 1;
+	struct perf_branch_entry *br = cpuc->lbr_entries;
 	u64 tos = intel_pmu_lbr_tos();
 	int i;
 
@@ -784,15 +785,11 @@ void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
 
 		rdmsrl(x86_pmu.lbr_from + lbr_idx, msr_lastbranch.lbr);
 
-		cpuc->lbr_entries[i].from	= msr_lastbranch.from;
-		cpuc->lbr_entries[i].to		= msr_lastbranch.to;
-		cpuc->lbr_entries[i].mispred	= 0;
-		cpuc->lbr_entries[i].predicted	= 0;
-		cpuc->lbr_entries[i].in_tx	= 0;
-		cpuc->lbr_entries[i].abort	= 0;
-		cpuc->lbr_entries[i].cycles	= 0;
-		cpuc->lbr_entries[i].type	= 0;
-		cpuc->lbr_entries[i].reserved	= 0;
+		perf_clear_branch_entry_bitfields(br);
+
+		br->from	= msr_lastbranch.from;
+		br->to		= msr_lastbranch.to;
+		br++;
 	}
 	cpuc->lbr_stack.nr = i;
 	cpuc->lbr_stack.hw_idx = tos;
@@ -807,6 +804,7 @@ void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)
 {
 	bool need_info = false, call_stack = false;
 	unsigned long mask = x86_pmu.lbr_nr - 1;
+	struct perf_branch_entry *br = cpuc->lbr_entries;
 	u64 tos = intel_pmu_lbr_tos();
 	int i;
 	int out = 0;
@@ -878,15 +876,14 @@ void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)
 		if (abort && x86_pmu.lbr_double_abort && out > 0)
 			out--;
 
-		cpuc->lbr_entries[out].from	 = from;
-		cpuc->lbr_entries[out].to	 = to;
-		cpuc->lbr_entries[out].mispred	 = mis;
-		cpuc->lbr_entries[out].predicted = pred;
-		cpuc->lbr_entries[out].in_tx	 = in_tx;
-		cpuc->lbr_entries[out].abort	 = abort;
-		cpuc->lbr_entries[out].cycles	 = cycles;
-		cpuc->lbr_entries[out].type	 = 0;
-		cpuc->lbr_entries[out].reserved	 = 0;
+		perf_clear_branch_entry_bitfields(br+out);
+		br[out].from	 = from;
+		br[out].to	 = to;
+		br[out].mispred	 = mis;
+		br[out].predicted = pred;
+		br[out].in_tx	 = in_tx;
+		br[out].abort	 = abort;
+		br[out].cycles	 = cycles;
 		out++;
 	}
 	cpuc->lbr_stack.nr = out;
@@ -951,6 +948,8 @@ static void intel_pmu_store_lbr(struct cpu_hw_events *cpuc,
 		to = rdlbr_to(i, lbr);
 		info = rdlbr_info(i, lbr);
 
+		perf_clear_branch_entry_bitfields(e);
+
 		e->from		= from;
 		e->to		= to;
 		e->mispred	= get_lbr_mispred(info);
@@ -959,7 +958,6 @@ static void intel_pmu_store_lbr(struct cpu_hw_events *cpuc,
 		e->abort	= !!(info & LBR_INFO_ABORT);
 		e->cycles	= get_lbr_cycles(info);
 		e->type		= get_lbr_br_type(info);
-		e->reserved	= 0;
 	}
 
 	cpuc->lbr_stack.nr = i;
include/linux/perf_event.h +16 -0
index af97dd4..a411080 100644
@@ -1063,6 +1063,22 @@ static inline void perf_sample_data_init(struct perf_sample_data *data,
 	data->txn = 0;
 }
 
+/*
+ * Clear all bitfields in the perf_branch_entry.
+ * The to and from fields are not cleared because they are
+ * systematically modified by caller.
+ */
+static inline void perf_clear_branch_entry_bitfields(struct perf_branch_entry *br)
+{
+	br->mispred = 0;
+	br->predicted = 0;
+	br->in_tx = 0;
+	br->abort = 0;
+	br->cycles = 0;
+	br->type = 0;
+	br->reserved = 0;
+}
+
 extern void perf_output_sample(struct perf_output_handle *handle,
 			       struct perf_event_header *header,
 			       struct perf_sample_data *data,
#3 lore raw

PATCH v7 2/13 x86/cpufeatures: add AMD Fam19h Branch Sampling feature

In reply to Stephane Eranian

Add a cpu feature for AMD Fam19h Branch Sampling feature as bit 31 of EBX on CPUID leaf function 0x80000008.

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index 9a59e2e73bee..ff3ee4f4ebbb 100644
@@ -315,6 +315,7 @@
 #define X86_FEATURE_VIRT_SSBD		(13*32+25) /* Virtualized Speculative Store Bypass Disable */
 #define X86_FEATURE_AMD_SSB_NO		(13*32+26) /* "" Speculative Store Bypass is fixed in hardware. */
 #define X86_FEATURE_CPPC		(13*32+27) /* Collaborative Processor Performance Control */
+#define X86_FEATURE_BRS			(13*32+31) /* Branch Sampling available */
 
 /* Thermal and Power Management Leaf, CPUID level 0x00000006 (EAX), word 14 */
 #define X86_FEATURE_DTHERM		(14*32+ 0) /* Digital Thermal Sensor */
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#4 lore raw

[tip: perf/core] x86/cpufeatures: Add AMD Fam19h Branch Sampling feature

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     a77d41ac3a0f41c80120ec5b8b08ab284fec950a
Gitweb:        https://git.kernel.org/tip/a77d41ac3a0f41c80120ec5b8b08ab284fec950a
Author:        Stephane Eranian <[email protected]>
AuthorDate:    Tue, 22 Mar 2022 15:15:06 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 05 Apr 2022 10:24:36 +02:00

x86/cpufeatures: Add AMD Fam19h Branch Sampling feature

Add a cpu feature for AMD Fam19h Branch Sampling feature as bit 31 of EBX on CPUID leaf function 0x80000008.

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index 73e643a..0d62afd 100644
@@ -315,6 +315,7 @@
 #define X86_FEATURE_VIRT_SSBD		(13*32+25) /* Virtualized Speculative Store Bypass Disable */
 #define X86_FEATURE_AMD_SSB_NO		(13*32+26) /* "" Speculative Store Bypass is fixed in hardware. */
 #define X86_FEATURE_CPPC		(13*32+27) /* Collaborative Processor Performance Control */
+#define X86_FEATURE_BRS			(13*32+31) /* Branch Sampling available */
 
 /* Thermal and Power Management Leaf, CPUID level 0x00000006 (EAX), word 14 */
 #define X86_FEATURE_DTHERM		(14*32+ 0) /* Digital Thermal Sensor */
#5 lore raw

PATCH v7 3/13 perf/x86/amd: add AMD Fam19h Branch Sampling support

In reply to Stephane Eranian

Add support for the AMD Fam19h 16-deep branch sampling feature as described in the AMD PPR Fam19h Model 01h Revision B1. This is a model specific extension. It is not an architected AMD feature.

The Branch Sampling (BRS) operates with a 16-deep saturating buffer in MSR registers. There is no branch type filtering. All control flow changes are captured. BRS relies on specific programming of the core PMU of Fam19h. In particular, the following requirements must be met:
- the sampling period be greater than 16 (BRS depth)
- the sampling period must use a fixed and not frequency mode

BRS interacts with the NMI interrupt as well. Because enabling BRS is expensive, it is only activated after P event occurrences, where P is the desired sampling period. At P occurrences of the event, the counter overflows, the CPU catches the interrupt, activates BRS for 16 branches until it saturates, and then delivers the NMI to the kernel. Between the overflow and the time BRS activates more branches may be executed skewing the period. All along, the sampling event keeps counting. The skid may be attenuated by reducing the sampling period by 16 (subsequent patch).

BRS is integrated into perf_events seamlessly via the same PERF_RECORD_BRANCH_STACK sample format. BRS generates perf_branch_entry records in the sampling buffer. No prediction information is supported. The branches are stored in reverse order of execution. The most recent branch is the first entry in each record.

No modification to the perf tool is necessary.

BRS can be used with any sampling event. However, it is recommended to use the RETIRED_BRANCH_INSTRUCTIONS event because it matches what the BRS captures.

$ perf record -b -c 1000037 -e cpu/event=0xc2,name=ret_br_instructions/ test
$ perf report -D
56531696056126 0x193c000 [0x1a8]: PERF_RECORD_SAMPLE(IP, 0x2): 18122/18230: 0x401d24 period: 1000037 addr: 0
... branch stack: nr:16
.....  0: 0000000000401d24 -> 0000000000401d5a 0 cycles      0
.....  1: 0000000000401d5c -> 0000000000401d24 0 cycles      0
.....  2: 0000000000401d22 -> 0000000000401d5c 0 cycles      0
.....  3: 0000000000401d5e -> 0000000000401d22 0 cycles      0
.....  4: 0000000000401d20 -> 0000000000401d5e 0 cycles      0
.....  5: 0000000000401d3e -> 0000000000401d20 0 cycles      0
.....  6: 0000000000401d42 -> 0000000000401d3e 0 cycles      0
.....  7: 0000000000401d3c -> 0000000000401d42 0 cycles      0
.....  8: 0000000000401d44 -> 0000000000401d3c 0 cycles      0
.....  9: 0000000000401d3a -> 0000000000401d44 0 cycles      0
..... 10: 0000000000401d46 -> 0000000000401d3a 0 cycles      0
..... 11: 0000000000401d38 -> 0000000000401d46 0 cycles      0
..... 12: 0000000000401d48 -> 0000000000401d38 0 cycles      0
..... 13: 0000000000401d36 -> 0000000000401d48 0 cycles      0
..... 14: 0000000000401d4a -> 0000000000401d36 0 cycles      0
..... 15: 0000000000401d34 -> 0000000000401d4a 0 cycles      0
 ... thread: test:18230
 ...... dso: test
---
 arch/x86/events/amd/Makefile     |   2 +-
 arch/x86/events/amd/brs.c        | 317 +++++++++++++++++++++++++++++++
 arch/x86/events/amd/core.c       | 233 ++++++++++++++++++++++-
 arch/x86/events/core.c           |  10 +-
 arch/x86/events/perf_event.h     | 101 ++++++++--
 arch/x86/include/asm/msr-index.h |   4 +
 6 files changed, 645 insertions(+), 22 deletions(-)
 create mode 100644 arch/x86/events/amd/brs.c
arch/x86/events/amd/Makefile +1 -1
index 6cbe38d5fd9d..cf323ffab5cd 100644
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_CPU_SUP_AMD)		+= core.o
+obj-$(CONFIG_CPU_SUP_AMD)		+= core.o brs.o
 obj-$(CONFIG_PERF_EVENTS_AMD_POWER)	+= power.o
 obj-$(CONFIG_X86_LOCAL_APIC)		+= ibs.o
 obj-$(CONFIG_PERF_EVENTS_AMD_UNCORE)	+= amd-uncore.o
arch/x86/events/amd/brs.c new file +317 -0

317 lines not on this page · raw message

arch/x86/events/amd/core.c +229 -4

322 lines not on this page · raw message

arch/x86/events/core.c +9 -1
index e686c5e0537b..c2a890caeb0a 100644
@@ -1338,6 +1338,10 @@ static void x86_pmu_enable(struct pmu *pmu)
 			if (hwc->state & PERF_HES_ARCH)
 				continue;
 
+			/*
+			 * if cpuc->enabled = 0, then no wrmsr as
+			 * per x86_pmu_enable_event()
+			 */
 			x86_pmu_start(event, PERF_EF_RELOAD);
 		}
 		cpuc->n_added = 0;
@@ -1704,11 +1708,15 @@ int x86_pmu_handle_irq(struct pt_regs *regs)
 		 * event overflow
 		 */
 		handled++;
-		perf_sample_data_init(&data, 0, event->hw.last_period);
 
 		if (!x86_perf_event_set_period(event))
 			continue;
 
+		perf_sample_data_init(&data, 0, event->hw.last_period);
+
+		if (has_branch_stack(event))
+			data.br_stack = &cpuc->lbr_stack;
+
 		if (perf_event_overflow(event, &data, regs))
 			x86_pmu_stop(event, 0);
 	}
arch/x86/events/perf_event.h +85 -16

131 lines not on this page · raw message

arch/x86/include/asm/msr-index.h +4 -0
index 87f80f0e5259..0179c72e829a 100644
@@ -688,6 +688,10 @@
 #define MSR_IA32_PERF_CTL		0x00000199
 #define INTEL_PERF_CTL_MASK		0xffff
 
+/* AMD Branch Sampling configuration */
+#define MSR_AMD_DBG_EXTN_CFG		0xc000010f
+#define MSR_AMD_SAMP_BR_FROM		0xc0010300
+
 #define MSR_IA32_MPERF			0x000000e7
 #define MSR_IA32_APERF			0x000000e8
 
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#6 lore raw
The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     ada543459cab7f653dcacdaba4011a8bb19c627c
Gitweb:        https://git.kernel.org/tip/ada543459cab7f653dcacdaba4011a8bb19c627c
Author:        Stephane Eranian <[email protected]>
AuthorDate:    Tue, 22 Mar 2022 15:15:07 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 05 Apr 2022 10:24:37 +02:00

perf/x86/amd: Add AMD Fam19h Branch Sampling support

Add support for the AMD Fam19h 16-deep branch sampling feature as described in the AMD PPR Fam19h Model 01h Revision B1. This is a model specific extension. It is not an architected AMD feature.

The Branch Sampling (BRS) operates with a 16-deep saturating buffer in MSR registers. There is no branch type filtering. All control flow changes are captured. BRS relies on specific programming of the core PMU of Fam19h. In particular, the following requirements must be met:
- the sampling period be greater than 16 (BRS depth)
- the sampling period must use a fixed and not frequency mode

BRS interacts with the NMI interrupt as well. Because enabling BRS is expensive, it is only activated after P event occurrences, where P is the desired sampling period. At P occurrences of the event, the counter overflows, the CPU catches the interrupt, activates BRS for 16 branches until it saturates, and then delivers the NMI to the kernel. Between the overflow and the time BRS activates more branches may be executed skewing the period. All along, the sampling event keeps counting. The skid may be attenuated by reducing the sampling period by 16 (subsequent patch).

BRS is integrated into perf_events seamlessly via the same PERF_RECORD_BRANCH_STACK sample format. BRS generates perf_branch_entry records in the sampling buffer. No prediction information is supported. The branches are stored in reverse order of execution. The most recent branch is the first entry in each record.

No modification to the perf tool is necessary.

BRS can be used with any sampling event. However, it is recommended to use the RETIRED_BRANCH_INSTRUCTIONS event because it matches what the BRS captures.

$ perf record -b -c 1000037 -e cpu/event=0xc2,name=ret_br_instructions/ test
$ perf report -D
56531696056126 0x193c000 [0x1a8]: PERF_RECORD_SAMPLE(IP, 0x2): 18122/18230: 0x401d24 period: 1000037 addr: 0
... branch stack: nr:16
.....  0: 0000000000401d24 -> 0000000000401d5a 0 cycles      0
.....  1: 0000000000401d5c -> 0000000000401d24 0 cycles      0
.....  2: 0000000000401d22 -> 0000000000401d5c 0 cycles      0
.....  3: 0000000000401d5e -> 0000000000401d22 0 cycles      0
.....  4: 0000000000401d20 -> 0000000000401d5e 0 cycles      0
.....  5: 0000000000401d3e -> 0000000000401d20 0 cycles      0
.....  6: 0000000000401d42 -> 0000000000401d3e 0 cycles      0
.....  7: 0000000000401d3c -> 0000000000401d42 0 cycles      0
.....  8: 0000000000401d44 -> 0000000000401d3c 0 cycles      0
.....  9: 0000000000401d3a -> 0000000000401d44 0 cycles      0
..... 10: 0000000000401d46 -> 0000000000401d3a 0 cycles      0
..... 11: 0000000000401d38 -> 0000000000401d46 0 cycles      0
..... 12: 0000000000401d48 -> 0000000000401d38 0 cycles      0
..... 13: 0000000000401d36 -> 0000000000401d48 0 cycles      0
..... 14: 0000000000401d4a -> 0000000000401d36 0 cycles      0
..... 15: 0000000000401d34 -> 0000000000401d4a 0 cycles      0
 ... thread: test:18230
 ...... dso: test
---
 arch/x86/events/amd/Makefile     |   2 +-
 arch/x86/events/amd/brs.c        | 317 ++++++++++++++++++++++++++++++-
 arch/x86/events/amd/core.c       | 233 +++++++++++++++++++++-
 arch/x86/events/core.c           |  10 +-
 arch/x86/events/perf_event.h     | 101 ++++++++--
 arch/x86/include/asm/msr-index.h |   4 +-
 6 files changed, 645 insertions(+), 22 deletions(-)
 create mode 100644 arch/x86/events/amd/brs.c
arch/x86/events/amd/Makefile +1 -1
index 6cbe38d..cf323ff 100644
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_CPU_SUP_AMD)		+= core.o
+obj-$(CONFIG_CPU_SUP_AMD)		+= core.o brs.o
 obj-$(CONFIG_PERF_EVENTS_AMD_POWER)	+= power.o
 obj-$(CONFIG_X86_LOCAL_APIC)		+= ibs.o
 obj-$(CONFIG_PERF_EVENTS_AMD_UNCORE)	+= amd-uncore.o
arch/x86/events/amd/brs.c new file +317 -0

317 lines not on this page · raw message

arch/x86/events/amd/core.c +229 -4

322 lines not on this page · raw message

arch/x86/events/core.c +9 -1
index eef816f..7ada917 100644
@@ -1338,6 +1338,10 @@ static void x86_pmu_enable(struct pmu *pmu)
 			if (hwc->state & PERF_HES_ARCH)
 				continue;
 
+			/*
+			 * if cpuc->enabled = 0, then no wrmsr as
+			 * per x86_pmu_enable_event()
+			 */
 			x86_pmu_start(event, PERF_EF_RELOAD);
 		}
 		cpuc->n_added = 0;
@@ -1704,11 +1708,15 @@ int x86_pmu_handle_irq(struct pt_regs *regs)
 		 * event overflow
 		 */
 		handled++;
-		perf_sample_data_init(&data, 0, event->hw.last_period);
 
 		if (!x86_perf_event_set_period(event))
 			continue;
 
+		perf_sample_data_init(&data, 0, event->hw.last_period);
+
+		if (has_branch_stack(event))
+			data.br_stack = &cpuc->lbr_stack;
+
 		if (perf_event_overflow(event, &data, regs))
 			x86_pmu_stop(event, 0);
 	}
arch/x86/events/perf_event.h +85 -16

131 lines not on this page · raw message

arch/x86/include/asm/msr-index.h +4 -0
index 0eb90d2..8179ea3 100644
@@ -688,6 +688,10 @@
 #define MSR_IA32_PERF_CTL		0x00000199
 #define INTEL_PERF_CTL_MASK		0xffff
 
+/* AMD Branch Sampling configuration */
+#define MSR_AMD_DBG_EXTN_CFG		0xc000010f
+#define MSR_AMD_SAMP_BR_FROM		0xc0010300
+
 #define MSR_IA32_MPERF			0x000000e7
 #define MSR_IA32_APERF			0x000000e8
 
#7 lore raw

PATCH v7 4/13 perf/x86/amd: add branch-brs helper event for Fam19h BRS

In reply to Stephane Eranian

Add a pseudo event called branch-brs to help use the FAM Fam19h Branch Sampling feature (BRS). BRS samples taken branches, so it is best used when sampling on a retired taken branch event (0xc4) which is what BRS captures. Instead of trying to remember the event code or actual event name, users can simply do:

$ perf record -b -e cpu/branch-brs/ -c 1000037 .....
---
 arch/x86/events/amd/core.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
arch/x86/events/amd/core.c +15 -0
index c7ac70d8ed9a..f7bce8364fe4 100644
@@ -1145,8 +1145,23 @@ static struct attribute_group group_caps_amd_brs = {
 	.is_visible = amd_brs_is_visible,
 };
 
+EVENT_ATTR_STR(branch-brs, amd_branch_brs,
+	       "event=" __stringify(AMD_FAM19H_BRS_EVENT)"\n");
+
+static struct attribute *amd_brs_events_attrs[] = {
+	EVENT_PTR(amd_branch_brs),
+	NULL,
+};
+
+static struct attribute_group group_events_amd_brs = {
+	.name       = "events",
+	.attrs      = amd_brs_events_attrs,
+	.is_visible = amd_brs_is_visible,
+};
+
 static const struct attribute_group *amd_attr_update[] = {
 	&group_caps_amd_brs,
+	&group_events_amd_brs,
 	NULL,
 };
 
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#8 lore raw

[tip: perf/core] perf/x86/amd: Add branch-brs helper event for Fam19h BRS

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     44175993efbae04e8b2d7f7795ff512c3a726db0
Gitweb:        https://git.kernel.org/tip/44175993efbae04e8b2d7f7795ff512c3a726db0
Author:        Stephane Eranian <[email protected]>
AuthorDate:    Tue, 22 Mar 2022 15:15:08 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 05 Apr 2022 10:24:37 +02:00

perf/x86/amd: Add branch-brs helper event for Fam19h BRS

Add a pseudo event called branch-brs to help use the FAM Fam19h Branch Sampling feature (BRS). BRS samples taken branches, so it is best used when sampling on a retired taken branch event (0xc4) which is what BRS captures. Instead of trying to remember the event code or actual event name, users can simply do:

$ perf record -b -e cpu/branch-brs/ -c 1000037 .....
---
 arch/x86/events/amd/core.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
arch/x86/events/amd/core.c +15 -0
index c7ac70d..f7bce83 100644
@@ -1145,8 +1145,23 @@ static struct attribute_group group_caps_amd_brs = {
 	.is_visible = amd_brs_is_visible,
 };
 
+EVENT_ATTR_STR(branch-brs, amd_branch_brs,
+	       "event=" __stringify(AMD_FAM19H_BRS_EVENT)"\n");
+
+static struct attribute *amd_brs_events_attrs[] = {
+	EVENT_PTR(amd_branch_brs),
+	NULL,
+};
+
+static struct attribute_group group_events_amd_brs = {
+	.name       = "events",
+	.attrs      = amd_brs_events_attrs,
+	.is_visible = amd_brs_is_visible,
+};
+
 static const struct attribute_group *amd_attr_update[] = {
 	&group_caps_amd_brs,
+	&group_events_amd_brs,
 	NULL,
 };
 
#9 lore raw

PATCH v7 5/13 perf/x86/amd: enable branch sampling priv level filtering

In reply to Stephane Eranian

The AMD Branch Sampling features does not provide hardware filtering by privilege level. The associated PMU counter does but not the branch sampling by itself. Given how BRS operates there is a possibility that BRS captures kernel level branches even though the event is programmed to count only at the user level.

Implement a workaround in software by removing the branches which belong to the wrong privilege level. The privilege level is evaluated on the target of the branch and not the source so as to be compatible with other architectures. As a consequence of this patch, the number of entries in the PERF_RECORD_BRANCH_STACK buffer may be less than the maximum (16). It could even be zero. Another consequence is that consecutive entries in the branch stack may not reflect actual code path and may have discontinuities, in case kernel branches were suppressed. But this is no different than what happens on other architectures.

---
 arch/x86/events/amd/brs.c | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)
arch/x86/events/amd/brs.c +20 -6
index 3c13c484c637..40461c3ce714 100644
@@ -92,10 +92,6 @@ int amd_brs_setup_filter(struct perf_event *event)
 	if ((type & ~PERF_SAMPLE_BRANCH_PLM_ALL) != PERF_SAMPLE_BRANCH_ANY)
 		return -EINVAL;
 
-	/* can only capture at all priv levels due to the way BRS works */
-	if ((type & PERF_SAMPLE_BRANCH_PLM_ALL) != PERF_SAMPLE_BRANCH_PLM_ALL)
-		return -EINVAL;
-
 	return 0;
 }
 
@@ -195,6 +191,21 @@ void amd_brs_disable_all(void)
 		amd_brs_disable();
 }
 
+static bool amd_brs_match_plm(struct perf_event *event, u64 to)
+{
+	int type = event->attr.branch_sample_type;
+	int plm_k = PERF_SAMPLE_BRANCH_KERNEL | PERF_SAMPLE_BRANCH_HV;
+	int plm_u = PERF_SAMPLE_BRANCH_USER;
+
+	if (!(type & plm_k) && kernel_ip(to))
+		return 0;
+
+	if (!(type & plm_u) && !kernel_ip(to))
+		return 0;
+
+	return 1;
+}
+
 /*
  * Caller must ensure amd_brs_inuse() is true before calling
  * return:
@@ -252,8 +263,6 @@ void amd_brs_drain(void)
 		if (to == BRS_POISON)
 			break;
 
-		rdmsrl(brs_from(brs_idx), from);
-
 		/*
 		 * Sign-extend SAMP_BR_TO to 64 bits, bits 61-63 are reserved.
 		 * Necessary to generate proper virtual addresses suitable for
@@ -261,6 +270,11 @@ void amd_brs_drain(void)
 		 */
 		to = (u64)(((s64)to << shift) >> shift);
 
+		if (!amd_brs_match_plm(event, to))
+			continue;
+
+		rdmsrl(brs_from(brs_idx), from);
+
 		perf_clear_branch_entry_bitfields(br+nr);
 
 		br[nr].from = from;
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#10 lore raw

[tip: perf/core] perf/x86/amd: Enable branch sampling priv level filtering

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     8910075d61a37e5b0d82e6c83ed9a0a31fe9ea08
Gitweb:        https://git.kernel.org/tip/8910075d61a37e5b0d82e6c83ed9a0a31fe9ea08
Author:        Stephane Eranian <[email protected]>
AuthorDate:    Tue, 22 Mar 2022 15:15:09 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 05 Apr 2022 10:24:37 +02:00

perf/x86/amd: Enable branch sampling priv level filtering

The AMD Branch Sampling features does not provide hardware filtering by privilege level. The associated PMU counter does but not the branch sampling by itself. Given how BRS operates there is a possibility that BRS captures kernel level branches even though the event is programmed to count only at the user level.

Implement a workaround in software by removing the branches which belong to the wrong privilege level. The privilege level is evaluated on the target of the branch and not the source so as to be compatible with other architectures. As a consequence of this patch, the number of entries in the PERF_RECORD_BRANCH_STACK buffer may be less than the maximum (16). It could even be zero. Another consequence is that consecutive entries in the branch stack may not reflect actual code path and may have discontinuities, in case kernel branches were suppressed. But this is no different than what happens on other architectures.

---
 arch/x86/events/amd/brs.c | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)
arch/x86/events/amd/brs.c +20 -6
index 3c13c48..40461c3 100644
@@ -92,10 +92,6 @@ int amd_brs_setup_filter(struct perf_event *event)
 	if ((type & ~PERF_SAMPLE_BRANCH_PLM_ALL) != PERF_SAMPLE_BRANCH_ANY)
 		return -EINVAL;
 
-	/* can only capture at all priv levels due to the way BRS works */
-	if ((type & PERF_SAMPLE_BRANCH_PLM_ALL) != PERF_SAMPLE_BRANCH_PLM_ALL)
-		return -EINVAL;
-
 	return 0;
 }
 
@@ -195,6 +191,21 @@ void amd_brs_disable_all(void)
 		amd_brs_disable();
 }
 
+static bool amd_brs_match_plm(struct perf_event *event, u64 to)
+{
+	int type = event->attr.branch_sample_type;
+	int plm_k = PERF_SAMPLE_BRANCH_KERNEL | PERF_SAMPLE_BRANCH_HV;
+	int plm_u = PERF_SAMPLE_BRANCH_USER;
+
+	if (!(type & plm_k) && kernel_ip(to))
+		return 0;
+
+	if (!(type & plm_u) && !kernel_ip(to))
+		return 0;
+
+	return 1;
+}
+
 /*
  * Caller must ensure amd_brs_inuse() is true before calling
  * return:
@@ -252,8 +263,6 @@ void amd_brs_drain(void)
 		if (to == BRS_POISON)
 			break;
 
-		rdmsrl(brs_from(brs_idx), from);
-
 		/*
 		 * Sign-extend SAMP_BR_TO to 64 bits, bits 61-63 are reserved.
 		 * Necessary to generate proper virtual addresses suitable for
@@ -261,6 +270,11 @@ void amd_brs_drain(void)
 		 */
 		to = (u64)(((s64)to << shift) >> shift);
 
+		if (!amd_brs_match_plm(event, to))
+			continue;
+
+		rdmsrl(brs_from(brs_idx), from);
+
 		perf_clear_branch_entry_bitfields(br+nr);
 
 		br[nr].from = from;
#11 lore raw

PATCH v7 6/13 perf/x86/amd: add AMD branch sampling period adjustment

In reply to Stephane Eranian

Add code to adjust the sampling event period when used with the Branch Sampling feature (BRS). Given the depth of the BRS (16), the period is reduced by that depth such that in the best case scenario, BRS saturates at the desired sampling period. In practice, though, the processor may execute more branches. Given a desired period P and a depth D, the kernel programs the actual period at P - D. After P occurrences of the sampling event, the counter overflows. It then may take X branches (skid) before the NMI is caught and held by the hardware and BRS activates. Then, after D branches, BRS saturates and the NMI is delivered. With no skid, the effective period would be (P - D) + D = P. In practice, however, it will likely be (P - D) + X + D. There is no way to eliminate X or predict X.

---
 arch/x86/events/core.c       |  7 +++++++
 arch/x86/events/perf_event.h | 12 ++++++++++++
 2 files changed, 19 insertions(+)
arch/x86/events/core.c +7 -0
index c2a890caeb0a..ed285f640efe 100644
@@ -1374,6 +1374,13 @@ int x86_perf_event_set_period(struct perf_event *event)
 	    x86_pmu.set_topdown_event_period)
 		return x86_pmu.set_topdown_event_period(event);
 
+	/*
+	 * decrease period by the depth of the BRS feature to get
+	 * the last N taken branches and approximate the desired period
+	 */
+	if (has_branch_stack(event))
+		period = amd_brs_adjust_period(period);
+
 	/*
 	 * If we are way outside a reasonable range then just skip forward:
 	 */
arch/x86/events/perf_event.h +12 -0
index 3485a4cf0241..25b037b571e4 100644
@@ -1263,6 +1263,14 @@ static inline bool amd_brs_active(void)
 	return cpuc->brs_active;
 }
 
+static inline s64 amd_brs_adjust_period(s64 period)
+{
+	if (period > x86_pmu.lbr_nr)
+		return period - x86_pmu.lbr_nr;
+
+	return period;
+}
+
 #else /* CONFIG_CPU_SUP_AMD */
 
 static inline int amd_pmu_init(void)
@@ -1287,6 +1295,10 @@ static inline void amd_brs_disable_all(void)
 {
 }
 
+static inline s64 amd_brs_adjust_period(s64 period)
+{
+	return period;
+}
 #endif /* CONFIG_CPU_SUP_AMD */
 
 static inline int is_pebs_pt(struct perf_event *event)
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#12 lore raw

[tip: perf/core] perf/x86/amd: Add AMD branch sampling period adjustment

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     ba2fe7500845a30fc845a72081999cf632051862
Gitweb:        https://git.kernel.org/tip/ba2fe7500845a30fc845a72081999cf632051862
Author:        Stephane Eranian <[email protected]>
AuthorDate:    Tue, 22 Mar 2022 15:15:10 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 05 Apr 2022 10:24:37 +02:00

perf/x86/amd: Add AMD branch sampling period adjustment

Add code to adjust the sampling event period when used with the Branch Sampling feature (BRS). Given the depth of the BRS (16), the period is reduced by that depth such that in the best case scenario, BRS saturates at the desired sampling period. In practice, though, the processor may execute more branches. Given a desired period P and a depth D, the kernel programs the actual period at P - D. After P occurrences of the sampling event, the counter overflows. It then may take X branches (skid) before the NMI is caught and held by the hardware and BRS activates. Then, after D branches, BRS saturates and the NMI is delivered. With no skid, the effective period would be (P - D) + D = P. In practice, however, it will likely be (P - D) + X + D. There is no way to eliminate X or predict X.

---
 arch/x86/events/core.c       |  7 +++++++
 arch/x86/events/perf_event.h | 12 ++++++++++++
 2 files changed, 19 insertions(+)
arch/x86/events/core.c +7 -0
index 7ada917..54f992e 100644
@@ -1375,6 +1375,13 @@ int x86_perf_event_set_period(struct perf_event *event)
 		return x86_pmu.set_topdown_event_period(event);
 
 	/*
+	 * decrease period by the depth of the BRS feature to get
+	 * the last N taken branches and approximate the desired period
+	 */
+	if (has_branch_stack(event))
+		period = amd_brs_adjust_period(period);
+
+	/*
 	 * If we are way outside a reasonable range then just skip forward:
 	 */
 	if (unlikely(left <= -period)) {
arch/x86/events/perf_event.h +12 -0
index 6f12651..d91ff2c 100644
@@ -1263,6 +1263,14 @@ static inline bool amd_brs_active(void)
 	return cpuc->brs_active;
 }
 
+static inline s64 amd_brs_adjust_period(s64 period)
+{
+	if (period > x86_pmu.lbr_nr)
+		return period - x86_pmu.lbr_nr;
+
+	return period;
+}
+
 #else /* CONFIG_CPU_SUP_AMD */
 
 static inline int amd_pmu_init(void)
@@ -1287,6 +1295,10 @@ static inline void amd_brs_disable_all(void)
 {
 }
 
+static inline s64 amd_brs_adjust_period(s64 period)
+{
+	return period;
+}
 #endif /* CONFIG_CPU_SUP_AMD */
 
 static inline int is_pebs_pt(struct perf_event *event)
#13 lore raw

PATCH v7 7/13 perf/x86/amd: make Zen3 branch sampling opt-in

In reply to Stephane Eranian

Add a kernel config option CONFIG_PERF_EVENTS_AMD_BRS to make the support for AMD Zen3 Branch Sampling (BRS) an opt-in compile time option.

---
 arch/x86/events/Kconfig      |  8 ++++++
 arch/x86/events/amd/Makefile |  3 ++-
 arch/x86/events/perf_event.h | 49 ++++++++++++++++++++++++++++--------
 3 files changed, 49 insertions(+), 11 deletions(-)
arch/x86/events/Kconfig +8 -0
index d6cdfe631674..09c56965750a 100644
@@ -44,4 +44,12 @@ config PERF_EVENTS_AMD_UNCORE
 
 	  To compile this driver as a module, choose M here: the
 	  module will be called 'amd-uncore'.
+
+config PERF_EVENTS_AMD_BRS
+	depends on PERF_EVENTS && CPU_SUP_AMD
+	bool "AMD Zen3 Branch Sampling support"
+	help
+	  Enable AMD Zen3 branch sampling support (BRS) which samples up to
+	  16 consecutive taken branches in registers.
+
 endmenu
arch/x86/events/amd/Makefile +2 -1
index cf323ffab5cd..b9f5d4610256 100644
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_CPU_SUP_AMD)		+= core.o brs.o
+obj-$(CONFIG_CPU_SUP_AMD)		+= core.o
+obj-$(CONFIG_PERF_EVENTS_AMD_BRS)	+= brs.o
 obj-$(CONFIG_PERF_EVENTS_AMD_POWER)	+= power.o
 obj-$(CONFIG_X86_LOCAL_APIC)		+= ibs.o
 obj-$(CONFIG_PERF_EVENTS_AMD_UNCORE)	+= amd-uncore.o
arch/x86/events/perf_event.h +39 -10

70 lines not on this page · raw message

Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#14 lore raw

[tip: perf/core] perf/x86/amd: Make Zen3 branch sampling opt-in

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     cc37e520a236069c0de0e7ea455082fa11c73b12
Gitweb:        https://git.kernel.org/tip/cc37e520a236069c0de0e7ea455082fa11c73b12
Author:        Stephane Eranian <[email protected]>
AuthorDate:    Tue, 22 Mar 2022 15:15:11 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 05 Apr 2022 10:24:38 +02:00

perf/x86/amd: Make Zen3 branch sampling opt-in

Add a kernel config option CONFIG_PERF_EVENTS_AMD_BRS to make the support for AMD Zen3 Branch Sampling (BRS) an opt-in compile time option.

---
 arch/x86/events/Kconfig      |  8 ++++++-
 arch/x86/events/amd/Makefile |  3 +-
 arch/x86/events/perf_event.h | 49 +++++++++++++++++++++++++++--------
 3 files changed, 49 insertions(+), 11 deletions(-)
arch/x86/events/Kconfig +8 -0
index d6cdfe6..09c5696 100644
@@ -44,4 +44,12 @@ config PERF_EVENTS_AMD_UNCORE
 
 	  To compile this driver as a module, choose M here: the
 	  module will be called 'amd-uncore'.
+
+config PERF_EVENTS_AMD_BRS
+	depends on PERF_EVENTS && CPU_SUP_AMD
+	bool "AMD Zen3 Branch Sampling support"
+	help
+	  Enable AMD Zen3 branch sampling support (BRS) which samples up to
+	  16 consecutive taken branches in registers.
+
 endmenu
arch/x86/events/amd/Makefile +2 -1
index cf323ff..b9f5d46 100644
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_CPU_SUP_AMD)		+= core.o brs.o
+obj-$(CONFIG_CPU_SUP_AMD)		+= core.o
+obj-$(CONFIG_PERF_EVENTS_AMD_BRS)	+= brs.o
 obj-$(CONFIG_PERF_EVENTS_AMD_POWER)	+= power.o
 obj-$(CONFIG_X86_LOCAL_APIC)		+= ibs.o
 obj-$(CONFIG_PERF_EVENTS_AMD_UNCORE)	+= amd-uncore.o
arch/x86/events/perf_event.h +39 -10

70 lines not on this page · raw message

#15 lore raw

PATCH v7 8/13 ACPI: add perf low power callback

In reply to Stephane Eranian

Add an optional callback needed by some PMU features, e.g., AMD BRS, to give a chance to the perf_events code to change its state before a CPU goes to low power and after it comes back.

The callback is void when the PERF_NEEDS_LOPWR_CB flag is not set. This flag must be set in arch specific perf_event.h header whenever needed. When not set, there is no impact on the ACPI code.

---
 drivers/acpi/acpi_pad.c       | 6 ++++++
 drivers/acpi/processor_idle.c | 5 +++++
 include/linux/perf_event.h    | 6 ++++++
 3 files changed, 17 insertions(+)
drivers/acpi/acpi_pad.c +6 -0
index f45979aa2d64..a306a07a60b5 100644
@@ -164,6 +164,9 @@ static int power_saving_thread(void *data)
 				tsc_marked_unstable = 1;
 			}
 			local_irq_disable();
+
+			perf_lopwr_cb(true);
+
 			tick_broadcast_enable();
 			tick_broadcast_enter();
 			stop_critical_timings();
@@ -172,6 +175,9 @@ static int power_saving_thread(void *data)
 
 			start_critical_timings();
 			tick_broadcast_exit();
+
+			perf_lopwr_cb(false);
+
 			local_irq_enable();
 
 			if (time_before(expire_time, jiffies)) {
drivers/acpi/processor_idle.c +5 -0
index f8e9fa82cb9b..f83596960d9a 100644
@@ -21,6 +21,7 @@
 #include <linux/cpuidle.h>
 #include <linux/cpu.h>
 #include <linux/minmax.h>
+#include <linux/perf_event.h>
 #include <acpi/processor.h>
 
 /*
@@ -549,6 +550,8 @@ static void wait_for_freeze(void)
  */
 static void __cpuidle acpi_idle_do_entry(struct acpi_processor_cx *cx)
 {
+	perf_lopwr_cb(true);
+
 	if (cx->entry_method == ACPI_CSTATE_FFH) {
 		/* Call into architectural FFH based C-state */
 		acpi_processor_ffh_cstate_enter(cx);
@@ -559,6 +562,8 @@ static void __cpuidle acpi_idle_do_entry(struct acpi_processor_cx *cx)
 		inb(cx->address);
 		wait_for_freeze();
 	}
+
+	perf_lopwr_cb(false);
 }
 
 /**
include/linux/perf_event.h +6 -0
index 496eb6aa6e54..1b98e46588bc 100644
@@ -1676,4 +1676,10 @@ typedef int (perf_snapshot_branch_stack_t)(struct perf_branch_entry *entries,
 					   unsigned int cnt);
 DECLARE_STATIC_CALL(perf_snapshot_branch_stack, perf_snapshot_branch_stack_t);
 
+#ifndef PERF_NEEDS_LOPWR_CB
+static inline void perf_lopwr_cb(bool mode)
+{
+}
+#endif
+
 #endif /* _LINUX_PERF_EVENT_H */
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#16 lore raw

[tip: perf/core] ACPI: Add perf low power callback

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     2a606a18cd672a16343d146a126721b34cc6adbd
Gitweb:        https://git.kernel.org/tip/2a606a18cd672a16343d146a126721b34cc6adbd
Author:        Stephane Eranian <[email protected]>
AuthorDate:    Tue, 22 Mar 2022 15:15:12 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 05 Apr 2022 10:24:38 +02:00

ACPI: Add perf low power callback

Add an optional callback needed by some PMU features, e.g., AMD BRS, to give a chance to the perf_events code to change its state before a CPU goes to low power and after it comes back.

The callback is void when the PERF_NEEDS_LOPWR_CB flag is not set. This flag must be set in arch specific perf_event.h header whenever needed. When not set, there is no impact on the ACPI code.

---
 drivers/acpi/acpi_pad.c       | 7 +++++++
 drivers/acpi/processor_idle.c | 5 +++++
 include/linux/perf_event.h    | 6 ++++++
 3 files changed, 18 insertions(+)
drivers/acpi/acpi_pad.c +7 -0
index f45979a..ec0e22a 100644
@@ -17,6 +17,7 @@
 #include <linux/tick.h>
 #include <linux/slab.h>
 #include <linux/acpi.h>
+#include <linux/perf_event.h>
 #include <asm/mwait.h>
 #include <xen/xen.h>
 
@@ -164,6 +165,9 @@ static int power_saving_thread(void *data)
 				tsc_marked_unstable = 1;
 			}
 			local_irq_disable();
+
+			perf_lopwr_cb(true);
+
 			tick_broadcast_enable();
 			tick_broadcast_enter();
 			stop_critical_timings();
@@ -172,6 +176,9 @@ static int power_saving_thread(void *data)
 
 			start_critical_timings();
 			tick_broadcast_exit();
+
+			perf_lopwr_cb(false);
+
 			local_irq_enable();
 
 			if (time_before(expire_time, jiffies)) {
drivers/acpi/processor_idle.c +5 -0
index 32b20ef..05dc0e1 100644
@@ -21,6 +21,7 @@
 #include <linux/cpuidle.h>
 #include <linux/cpu.h>
 #include <linux/minmax.h>
+#include <linux/perf_event.h>
 #include <acpi/processor.h>
 
 /*
@@ -549,6 +550,8 @@ static void wait_for_freeze(void)
  */
 static void __cpuidle acpi_idle_do_entry(struct acpi_processor_cx *cx)
 {
+	perf_lopwr_cb(true);
+
 	if (cx->entry_method == ACPI_CSTATE_FFH) {
 		/* Call into architectural FFH based C-state */
 		acpi_processor_ffh_cstate_enter(cx);
@@ -559,6 +562,8 @@ static void __cpuidle acpi_idle_do_entry(struct acpi_processor_cx *cx)
 		inb(cx->address);
 		wait_for_freeze();
 	}
+
+	perf_lopwr_cb(false);
 }
 
 /**
include/linux/perf_event.h +6 -0
index a411080..da75956 100644
@@ -1676,4 +1676,10 @@ typedef int (perf_snapshot_branch_stack_t)(struct perf_branch_entry *entries,
 					   unsigned int cnt);
 DECLARE_STATIC_CALL(perf_snapshot_branch_stack, perf_snapshot_branch_stack_t);
 
+#ifndef PERF_NEEDS_LOPWR_CB
+static inline void perf_lopwr_cb(bool mode)
+{
+}
+#endif
+
 #endif /* _LINUX_PERF_EVENT_H */
#17 lore raw

PATCH v7 9/13 perf/x86/amd: add idle hooks for branch sampling

In reply to Stephane Eranian

On AMD Fam19h Zen3, the branch sampling (BRS) feature must be disabled before entering low power and re-enabled (if was active) when returning from low power. Otherwise, the NMI interrupt may be held up for too long and cause problems. Stopping BRS will cause the NMI to be delivered if it was held up.

Define a perf_amd_brs_lopwr_cb() callback to stop/restart BRS. The callback is protected by a jump label which is enabled only when AMD BRS is detected. In all other cases, the callback is never called.

---
 arch/x86/events/amd/brs.c         | 32 +++++++++++++++++++++++++++++++
 arch/x86/events/amd/core.c        |  4 ++++
 arch/x86/events/perf_event.h      |  1 +
 arch/x86/include/asm/perf_event.h | 21 ++++++++++++++++++++
 4 files changed, 58 insertions(+)
arch/x86/events/amd/brs.c +32 -0
index 40461c3ce714..185a58cea917 100644
@@ -7,6 +7,7 @@
  * Contributed by Stephane Eranian <[email protected]>
  */
 #include <linux/kernel.h>
+#include <linux/jump_label.h>
 #include <asm/msr.h>
 #include <asm/cpufeature.h>
 
@@ -329,3 +330,34 @@ void amd_pmu_brs_sched_task(struct perf_event_context *ctx, bool sched_in)
 	if (sched_in)
 		amd_brs_poison_buffer();
 }
+
+DEFINE_STATIC_KEY_FALSE(perf_lopwr_needed);
+
+/*
+ * called from ACPI processor_idle.c or acpi_pad.c
+ * with interrupts disabled
+ */
+void perf_amd_brs_lopwr_cb(bool lopwr_in)
+{
+	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+	union amd_debug_extn_cfg cfg;
+
+	/*
+	 * on mwait in, we may end up in non C0 state.
+	 * we must disable branch sampling to avoid holding the NMI
+	 * for too long. We disable it in hardware but we
+	 * keep the state in cpuc, so we can re-enable.
+	 *
+	 * The hardware will deliver the NMI if needed when brsmen cleared
+	 */
+	if (cpuc->brs_active) {
+		cfg.val = get_debug_extn_cfg();
+		cfg.brsmen = !lopwr_in;
+		set_debug_extn_cfg(cfg.val);
+	}
+}
+
+void __init amd_brs_lopwr_init(void)
+{
+	static_branch_enable(&perf_lopwr_needed);
+}
arch/x86/events/amd/core.c +4 -0
index f7bce8364fe4..8e1e818f8195 100644
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0-only
 #include <linux/perf_event.h>
+#include <linux/jump_label.h>
 #include <linux/export.h>
 #include <linux/types.h>
 #include <linux/init.h>
@@ -1225,6 +1226,9 @@ static int __init amd_core_pmu_init(void)
 		/*
 		 * put_event_constraints callback same as Fam17h, set above
 		 */
+
+		/* branch sampling must be stopped when entering low power */
+		amd_brs_lopwr_init();
 	}
 
 	x86_pmu.attr_update = amd_attr_update;
arch/x86/events/perf_event.h +1 -0
index 4d050579dcbd..2ed7bf5b51b1 100644
@@ -1226,6 +1226,7 @@ void amd_brs_enable(void);
 void amd_brs_enable_all(void);
 void amd_brs_disable_all(void);
 void amd_brs_drain(void);
+void amd_brs_lopwr_init(void);
 void amd_brs_disable_all(void);
 int amd_brs_setup_filter(struct perf_event *event);
 void amd_brs_reset(void);
arch/x86/include/asm/perf_event.h +21 -0
index 58d9e4b1fa0a..42753a9dc3ed 100644
@@ -513,6 +513,27 @@ static inline void intel_pt_handle_vmx(int on)
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_AMD)
  extern void amd_pmu_enable_virt(void);
  extern void amd_pmu_disable_virt(void);
+
+#if defined(CONFIG_PERF_EVENTS_AMD_BRS)
+
+#define PERF_NEEDS_LOPWR_CB 1
+
+/*
+ * architectural low power callback impacts
+ * drivers/acpi/processor_idle.c
+ * drivers/acpi/acpi_pad.c
+ */
+extern void perf_amd_brs_lopwr_cb(bool lopwr_in);
+DECLARE_STATIC_KEY_FALSE(perf_lopwr_needed);
+
+static inline void perf_lopwr_cb(bool mode)
+{
+	/* key enabled only when BRS is available */
+	if (static_branch_unlikely(&perf_lopwr_needed))
+		perf_amd_brs_lopwr_cb(mode);
+}
+#endif /* PERF_NEEDS_LOPWR_CB */
+
 #else
  static inline void amd_pmu_enable_virt(void) { }
  static inline void amd_pmu_disable_virt(void) { }
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#18 lore raw

[tip: perf/core] perf/x86/amd: Add idle hooks for branch sampling

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     d5616bac7adadbf42a3b63b8717e75eb82a2cc2c
Gitweb:        https://git.kernel.org/tip/d5616bac7adadbf42a3b63b8717e75eb82a2cc2c
Author:        Stephane Eranian <[email protected]>
AuthorDate:    Tue, 22 Mar 2022 15:15:13 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 05 Apr 2022 10:24:38 +02:00

perf/x86/amd: Add idle hooks for branch sampling

On AMD Fam19h Zen3, the branch sampling (BRS) feature must be disabled before entering low power and re-enabled (if was active) when returning from low power. Otherwise, the NMI interrupt may be held up for too long and cause problems. Stopping BRS will cause the NMI to be delivered if it was held up.

Define a perf_amd_brs_lopwr_cb() callback to stop/restart BRS. The callback is protected by a jump label which is enabled only when AMD BRS is detected. In all other cases, the callback is never called.

---
 arch/x86/events/amd/brs.c         | 33 ++++++++++++++++++++++++++++++-
 arch/x86/events/amd/core.c        |  4 ++++-
 arch/x86/events/perf_event.h      |  1 +-
 arch/x86/include/asm/perf_event.h | 23 +++++++++++++++++++++-
 4 files changed, 61 insertions(+)
arch/x86/events/amd/brs.c +33 -0
index 40461c3..895c821 100644
@@ -7,6 +7,7 @@
  * Contributed by Stephane Eranian <[email protected]>
  */
 #include <linux/kernel.h>
+#include <linux/jump_label.h>
 #include <asm/msr.h>
 #include <asm/cpufeature.h>
 
@@ -329,3 +330,35 @@ void amd_pmu_brs_sched_task(struct perf_event_context *ctx, bool sched_in)
 	if (sched_in)
 		amd_brs_poison_buffer();
 }
+
+/*
+ * called from ACPI processor_idle.c or acpi_pad.c
+ * with interrupts disabled
+ */
+void perf_amd_brs_lopwr_cb(bool lopwr_in)
+{
+	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+	union amd_debug_extn_cfg cfg;
+
+	/*
+	 * on mwait in, we may end up in non C0 state.
+	 * we must disable branch sampling to avoid holding the NMI
+	 * for too long. We disable it in hardware but we
+	 * keep the state in cpuc, so we can re-enable.
+	 *
+	 * The hardware will deliver the NMI if needed when brsmen cleared
+	 */
+	if (cpuc->brs_active) {
+		cfg.val = get_debug_extn_cfg();
+		cfg.brsmen = !lopwr_in;
+		set_debug_extn_cfg(cfg.val);
+	}
+}
+
+DEFINE_STATIC_CALL_NULL(perf_lopwr_cb, perf_amd_brs_lopwr_cb);
+EXPORT_STATIC_CALL_TRAMP_GPL(perf_lopwr_cb);
+
+void __init amd_brs_lopwr_init(void)
+{
+	static_call_update(perf_lopwr_cb, perf_amd_brs_lopwr_cb);
+}
arch/x86/events/amd/core.c +4 -0
index f7bce83..8e1e818 100644
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0-only
 #include <linux/perf_event.h>
+#include <linux/jump_label.h>
 #include <linux/export.h>
 #include <linux/types.h>
 #include <linux/init.h>
@@ -1225,6 +1226,9 @@ static int __init amd_core_pmu_init(void)
 		/*
 		 * put_event_constraints callback same as Fam17h, set above
 		 */
+
+		/* branch sampling must be stopped when entering low power */
+		amd_brs_lopwr_init();
 	}
 
 	x86_pmu.attr_update = amd_attr_update;
arch/x86/events/perf_event.h +1 -0
index ef27aee..3b03245 100644
@@ -1226,6 +1226,7 @@ void amd_brs_enable(void);
 void amd_brs_enable_all(void);
 void amd_brs_disable_all(void);
 void amd_brs_drain(void);
+void amd_brs_lopwr_init(void);
 void amd_brs_disable_all(void);
 int amd_brs_setup_filter(struct perf_event *event);
 void amd_brs_reset(void);
arch/x86/include/asm/perf_event.h +23 -0
index 58d9e4b..8199fc5 100644
@@ -2,6 +2,8 @@
 #ifndef _ASM_X86_PERF_EVENT_H
 #define _ASM_X86_PERF_EVENT_H
 
+#include <linux/static_call.h>
+
 /*
  * Performance event hw details:
  */
@@ -513,6 +515,27 @@ static inline void intel_pt_handle_vmx(int on)
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_AMD)
  extern void amd_pmu_enable_virt(void);
  extern void amd_pmu_disable_virt(void);
+
+#if defined(CONFIG_PERF_EVENTS_AMD_BRS)
+
+#define PERF_NEEDS_LOPWR_CB 1
+
+/*
+ * architectural low power callback impacts
+ * drivers/acpi/processor_idle.c
+ * drivers/acpi/acpi_pad.c
+ */
+extern void perf_amd_brs_lopwr_cb(bool lopwr_in);
+
+DECLARE_STATIC_CALL(perf_lopwr_cb, perf_amd_brs_lopwr_cb);
+
+static inline void perf_lopwr_cb(bool lopwr_in)
+{
+	static_call_mod(perf_lopwr_cb)(lopwr_in);
+}
+
+#endif /* PERF_NEEDS_LOPWR_CB */
+
 #else
  static inline void amd_pmu_enable_virt(void) { }
  static inline void amd_pmu_disable_virt(void) { }
#19 lore raw

PATCH v7 10/13 perf tools: fix NULL point in evsel__env()

In reply to Stephane Eranian

evsel_env() cannot return NULL. In case evsel->evlist->env is NULL default to perf_env. Fixes crashes in evsel__open_strerror()

---
 tools/perf/util/evsel.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/evsel.c +1 -1
index 22d3267ce294..14b0e7ffa2c7 100644
@@ -2965,7 +2965,7 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,
 
 struct perf_env *evsel__env(struct evsel *evsel)
 {
-	if (evsel && evsel->evlist)
+	if (evsel && evsel->evlist && evsel->evlist->env)
 		return evsel->evlist->env;
 	return &perf_env;
 }
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#20 lore raw

Re: [PATCH v7 10/13] perf tools: fix NULL point in evsel__env()

Quoted 6 lines · Stephane Eranian

evsel_env() cannot return NULL. In case evsel->evlist->env is NULL default to perf_env. Fixes crashes in evsel__open_strerror()

---

FWIW, here's a link to one of the original submissions:

https://lkml.org/lkml/2022/1/26/1220

Thanks,

Kim

#21 lore raw

Re: [PATCH v7 10/13] perf tools: fix NULL point in evsel__env()

In reply to Stephane Eranian

Em Tue, Mar 22, 2022 at 03:15:14PM -0700, Stephane Eranian escreveu:

Quoted 5 lines

evsel_env() cannot return NULL. In case evsel->evlist->env is NULL default to perf_env. Fixes crashes in evsel__open_strerror()

I have this already in from a patch by Kim Phillips.

- Arnaldo

Quoted 19 lines
---
 tools/perf/util/evsel.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/evsel.c +1 -1
index 22d3267ce294..14b0e7ffa2c7 100644
@@ -2965,7 +2965,7 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,
 
 struct perf_env *evsel__env(struct evsel *evsel)
 {
-	if (evsel && evsel->evlist)
+	if (evsel && evsel->evlist && evsel->evlist->env)
 		return evsel->evlist->env;
 	return &perf_env;
 }
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
Signature · 1 line
- Arnaldo
#22 lore raw

Re: [PATCH v7 10/13] perf tools: fix NULL point in evsel__env()

Quoted 10 lines · Arnaldo Carvalho de Melo

Em Tue, Mar 22, 2022 at 03:15:14PM -0700, Stephane Eranian escreveu:

Quoted 5 lines

evsel_env() cannot return NULL. In case evsel->evlist->env is NULL default to perf_env. Fixes crashes in evsel__open_strerror()

I have this already in from a patch by Kim Phillips.

Ok, then. We are all set.

Quoted 25 lines

- Arnaldo

Quoted 18 lines
---
 tools/perf/util/evsel.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/evsel.c +1 -2
index 22d3267ce294..14b0e7ffa2c7 100644
@@ -2965,7 +2965,7 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,
 
 struct perf_env *evsel__env(struct evsel *evsel)
 {
-     if (evsel && evsel->evlist)
+     if (evsel && evsel->evlist && evsel->evlist->env)
              return evsel->evlist->env;
      return &perf_env;
 }
--
Quoted 1 line

2.35.1.894.gb6a874cedc-goog

--

- Arnaldo

#23 lore raw

PATCH v7 11/13 perf tools: Improve IBS error handling

In reply to Stephane Eranian

From: Kim Phillips <[email protected]>

improve the error message returned on failed perf_event_open() on AMD when using IBS.

Output of executing 'perf record -e ibs_op// true' BEFORE this patch:

The sys_perf_event_open() syscall returned with 22 (Invalid argument)for event (ibs_op//u). /bin/dmesg | grep -i perf may provide additional information.

Output after:

AMD IBS cannot exclude kernel events. Try running at a higher privilege level.

Output of executing 'sudo perf record -e ibs_op// true' BEFORE this patch:

Error:
The sys_perf_event_open() syscall returned with 22 (Invalid argument) for event (ibs_op//). /bin/dmesg | grep -i perf may provide additional information.

Output after:

Error:
AMD IBS may only be available in system-wide/per-cpu mode. Try using -a, or -C and workload affinity

Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Boris Ostrovsky <[email protected]>
Cc Ian Rogers <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Joao Martins <[email protected]>
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Michael Petlan <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Robert Richter <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 tools/perf/util/evsel.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
tools/perf/util/evsel.c +25 -0
index 14b0e7ffa2c7..e8ff7a4bd490 100644
@@ -2847,9 +2847,23 @@ static bool find_process(const char *name)
 	return ret ? false : true;
 }
 
+static bool is_amd(const char *arch, const char *cpuid)
+{
+	return arch && !strcmp("x86", arch) && cpuid && strstarts(cpuid, "AuthenticAMD");
+}
+
+static bool is_amd_ibs(struct evsel *evsel)
+{
+	return evsel->core.attr.precise_ip
+	    || (evsel->pmu_name && !strncmp(evsel->pmu_name, "ibs", 3));
+}
+
 int evsel__open_strerror(struct evsel *evsel, struct target *target,
 			 int err, char *msg, size_t size)
 {
+	struct perf_env *env = evsel__env(evsel);
+	const char *arch = perf_env__arch(env);
+	const char *cpuid = perf_env__cpuid(env);
 	char sbuf[STRERR_BUFSIZE];
 	int printed = 0, enforced = 0;
 
@@ -2949,6 +2963,17 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,
 			return scnprintf(msg, size,
 	"Invalid event (%s) in per-thread mode, enable system wide with '-a'.",
 					evsel__name(evsel));
+		if (is_amd(arch, cpuid)) {
+			if (is_amd_ibs(evsel)) {
+				if (evsel->core.attr.exclude_kernel)
+					return scnprintf(msg, size,
+	"AMD IBS can't exclude kernel events.  Try running at a higher privilege level.");
+				if (!evsel->core.system_wide)
+					return scnprintf(msg, size,
+	"AMD IBS may only be available in system-wide/per-cpu mode.  Try using -a, or -C and workload affinity");
+			}
+		}
+
 		break;
 	case ENODATA:
 		return scnprintf(msg, size, "Cannot collect data source with the load latency event alone. "
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#24 lore raw

Re: [PATCH v7 11/13] perf tools: Improve IBS error handling

Em Tue, Mar 22, 2022 at 03:15:15PM -0700, Stephane Eranian escreveu:

Quoted 5 lines

From: Kim Phillips <[email protected]>

improve the error message returned on failed perf_event_open() on AMD when using IBS.

Quoted 8 lines

Output of executing 'perf record -e ibs_op// true' BEFORE this patch:

The sys_perf_event_open() syscall returned with 22 (Invalid argument)for event (ibs_op//u). /bin/dmesg | grep -i perf may provide additional information.

Output after:

AMD IBS cannot exclude kernel events. Try running at a higher privilege level.

So the error message don't match up the BEFORE part, that didn't have a "u" at the end, and with this patch in:

  $ git log --oneline -1
  ca585f91ce13df4c (HEAD -> perf/core) perf evsel: Improve AMD IBS (Instruction-Based Sampling) error handling messages
  # perf -v
  perf version 5.17.gca585f91ce13
  #

On a:

  # grep -m1 "model name" /proc/cpuinfo
  model name	: AMD Ryzen 9 5950X 16-Core Processor
  #

If I try:

  # perf record -e ibs_op//u true
  Error:
  Invalid event (ibs_op//u) in per-thread mode, enable system wide with '-a'.
  # 

So now if I try:

  # perf record -a -e ibs_op//u true
  Error:
  AMD IBS can't exclude kernel events.  Try running at a higher privilege level.
  #

So the problem is with the patch description, I'm fixing it.

Quoted 10 lines

Output of executing 'sudo perf record -e ibs_op// true' BEFORE this patch:

Error:
The sys_perf_event_open() syscall returned with 22 (Invalid argument) for event (ibs_op//). /bin/dmesg | grep -i perf may provide additional information.

Output after:

Error:
AMD IBS may only be available in system-wide/per-cpu mode. Try using -a, or -C and workload affinity

But this one is never reached:

[root@five ~]# perf record -e ibs_op// true
Error:
Invalid event (ibs_op//) in per-thread mode, enable system wide with '-a'

tools/perf/util/evsel.c int evsel__open_strerror(struct evsel *evsel, struct target *target, · line 2973

Quoted 3 lines
tools/perf/util/evsel.c +3 -0
+                             if (!evsel->core.system_wide)
+                                     return scnprintf(msg, size,
+     "AMD IBS may only be available in system-wide/per-cpu mode.  Try using -a, or -C and workload affinity");

I'm applying this to make progress, the message now provides more clues.

- Arnaldo

Quoted 67 lines
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Boris Ostrovsky <[email protected]>
Cc Ian Rogers <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Joao Martins <[email protected]>
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Michael Petlan <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Robert Richter <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 tools/perf/util/evsel.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
tools/perf/util/evsel.c +25 -0
index 14b0e7ffa2c7..e8ff7a4bd490 100644
@@ -2847,9 +2847,23 @@ static bool find_process(const char *name)
 	return ret ? false : true;
 }
 
+static bool is_amd(const char *arch, const char *cpuid)
+{
+	return arch && !strcmp("x86", arch) && cpuid && strstarts(cpuid, "AuthenticAMD");
+}
+
+static bool is_amd_ibs(struct evsel *evsel)
+{
+	return evsel->core.attr.precise_ip
+	    || (evsel->pmu_name && !strncmp(evsel->pmu_name, "ibs", 3));
+}
+
 int evsel__open_strerror(struct evsel *evsel, struct target *target,
 			 int err, char *msg, size_t size)
 {
+	struct perf_env *env = evsel__env(evsel);
+	const char *arch = perf_env__arch(env);
+	const char *cpuid = perf_env__cpuid(env);
 	char sbuf[STRERR_BUFSIZE];
 	int printed = 0, enforced = 0;
 
@@ -2949,6 +2963,17 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,
 			return scnprintf(msg, size,
 	"Invalid event (%s) in per-thread mode, enable system wide with '-a'.",
 					evsel__name(evsel));
+		if (is_amd(arch, cpuid)) {
+			if (is_amd_ibs(evsel)) {
+				if (evsel->core.attr.exclude_kernel)
+					return scnprintf(msg, size,
+	"AMD IBS can't exclude kernel events.  Try running at a higher privilege level.");
+				if (!evsel->core.system_wide)
+					return scnprintf(msg, size,
+	"AMD IBS may only be available in system-wide/per-cpu mode.  Try using -a, or -C and workload affinity");
+			}
+		}
+
 		break;
 	case ENODATA:
 		return scnprintf(msg, size, "Cannot collect data source with the load latency event alone. "
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
Signature · 1 line
- Arnaldo
maintainer Tested-by
#25 lore raw

Re: [PATCH v7 11/13] perf tools: Improve IBS error handling

Em Fri, Mar 25, 2022 at 06:46:08PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 11 lines

Em Tue, Mar 22, 2022 at 03:15:15PM -0700, Stephane Eranian escreveu:

Quoted 10 lines

From: Kim Phillips <[email protected]>

improve the error message returned on failed perf_event_open() on AMD when using IBS.

Output of executing 'perf record -e ibs_op// true' BEFORE this patch:

The sys_perf_event_open() syscall returned with 22 (Invalid argument)for event (ibs_op//u). /bin/dmesg | grep -i perf may provide additional information.

Quoted 6 lines
Quoted 3 lines

Output after:

AMD IBS cannot exclude kernel events. Try running at a higher privilege level.

So the error message don't match up the BEFORE part, that didn't have a "u" at the end, and with this patch in:

I see, the patch description doesn't mention that the BEFORE command is run as a NON root user, when the 'u' suffix is auto-added by perf, I'm ammending the comment log message with this.

Now it reads:

----------------------------------------------------------------------
perf evsel: Improve AMD IBS (Instruction-Based Sampling) error handling messages

Improve the error message returned on failed perf_event_open() on AMD systems when using IBS (Instruction-Based Sampling).

Output of executing 'perf record -e ibs_op// true' as a non root user BEFORE this patch (perf will add the 'u' modifier at the end to exclude kernel/hypervisor sampling):

The sys_perf_event_open() syscall returned with 22 (Invalid argument)for event (ibs_op//u). /bin/dmesg | grep -i perf may provide additional information.

Output after:

AMD IBS can't exclude kernel events. Try running at a higher privilege level.

Output of executing 'sudo perf record -e ibs_op// true' BEFORE this patch:

Error:
The sys_perf_event_open() syscall returned with 22 (Invalid argument) for event (ibs_op//). /bin/dmesg | grep -i perf may provide additional information.

Output after:

Error:
Invalid event (ibs_op//) in per-thread mode, enable system wide with '-a'.

Folowing the suggestion:

  $ sudo perf record -a -e ibs_op// true
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 1.664 MB perf.data (194 samples) ]
  $

----------------------------------------------------------------------

This is all with:

[root@five ~]# uname -a
Linux five 5.16.16-200.fc35.x86_64 #1 SMP PREEMPT Sat Mar 19 13:52:41 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

- Arnaldo

Quoted 126 lines
  $ git log --oneline -1
  ca585f91ce13df4c (HEAD -> perf/core) perf evsel: Improve AMD IBS (Instruction-Based Sampling) error handling messages
  # perf -v
  perf version 5.17.gca585f91ce13
  #

On a:

  # grep -m1 "model name" /proc/cpuinfo
  model name	: AMD Ryzen 9 5950X 16-Core Processor
  #

If I try:

  # perf record -e ibs_op//u true
  Error:
  Invalid event (ibs_op//u) in per-thread mode, enable system wide with '-a'.
  # 

So now if I try:

  # perf record -a -e ibs_op//u true
  Error:
  AMD IBS can't exclude kernel events.  Try running at a higher privilege level.
  #

So the problem is with the patch description, I'm fixing it.

Quoted 10 lines

Output of executing 'sudo perf record -e ibs_op// true' BEFORE this patch:

Error:
The sys_perf_event_open() syscall returned with 22 (Invalid argument) for event (ibs_op//). /bin/dmesg | grep -i perf may provide additional information.

Output after:

Error:
AMD IBS may only be available in system-wide/per-cpu mode. Try using -a, or -C and workload affinity

But this one is never reached:

[root@five ~]# perf record -e ibs_op// true
Error:
Invalid event (ibs_op//) in per-thread mode, enable system wide with '-a'

tools/perf/util/evsel.c int evsel__open_strerror(struct evsel *evsel, struct target *target, · line 2973

Quoted 3 lines
tools/perf/util/evsel.c +3 -0
+                             if (!evsel->core.system_wide)
+                                     return scnprintf(msg, size,
+     "AMD IBS may only be available in system-wide/per-cpu mode.  Try using -a, or -C and workload affinity");

I'm applying this to make progress, the message now provides more clues.

- Arnaldo

Quoted 67 lines
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Boris Ostrovsky <[email protected]>
Cc Ian Rogers <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Joao Martins <[email protected]>
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Michael Petlan <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Robert Richter <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 tools/perf/util/evsel.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
tools/perf/util/evsel.c +25 -0
index 14b0e7ffa2c7..e8ff7a4bd490 100644
@@ -2847,9 +2847,23 @@ static bool find_process(const char *name)
 	return ret ? false : true;
 }
 
+static bool is_amd(const char *arch, const char *cpuid)
+{
+	return arch && !strcmp("x86", arch) && cpuid && strstarts(cpuid, "AuthenticAMD");
+}
+
+static bool is_amd_ibs(struct evsel *evsel)
+{
+	return evsel->core.attr.precise_ip
+	    || (evsel->pmu_name && !strncmp(evsel->pmu_name, "ibs", 3));
+}
+
 int evsel__open_strerror(struct evsel *evsel, struct target *target,
 			 int err, char *msg, size_t size)
 {
+	struct perf_env *env = evsel__env(evsel);
+	const char *arch = perf_env__arch(env);
+	const char *cpuid = perf_env__cpuid(env);
 	char sbuf[STRERR_BUFSIZE];
 	int printed = 0, enforced = 0;
 
@@ -2949,6 +2963,17 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,
 			return scnprintf(msg, size,
 	"Invalid event (%s) in per-thread mode, enable system wide with '-a'.",
 					evsel__name(evsel));
+		if (is_amd(arch, cpuid)) {
+			if (is_amd_ibs(evsel)) {
+				if (evsel->core.attr.exclude_kernel)
+					return scnprintf(msg, size,
+	"AMD IBS can't exclude kernel events.  Try running at a higher privilege level.");
+				if (!evsel->core.system_wide)
+					return scnprintf(msg, size,
+	"AMD IBS may only be available in system-wide/per-cpu mode.  Try using -a, or -C and workload affinity");
+			}
+		}
+
 		break;
 	case ENODATA:
 		return scnprintf(msg, size, "Cannot collect data source with the load latency event alone. "
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
Signature · 1 line
- Arnaldo
Signature · 1 line
- Arnaldo
#26 lore raw

PATCH v7 12/13 perf tools: Improve error handling of AMD Branch Sampling

In reply to Stephane Eranian

Improve the error message printed by perf when perf_event_open() fails on AMD Zen3 when using the branch sampling feature. In the case of EINVAL, there are two main reasons: frequency mode or period is smaller than the depth of the branch sampling buffer (16). The patch checks the parameters of the call and tries to print a relevant message to explain the error:

$ perf record -b -e cpu/branch-brs/ -c 10 ls
Error:
AMD Branch Sampling does not support sampling period smaller than what is reported in /sys/devices/cpu/caps/branches.
$ perf record -b -e cpu/branch-brs/ ls
Error:
AMD Branch Sampling does not support frequency mode sampling, must pass a fixed sampling period via -c option or cpu/branch-brs,period=xxxx/.
[Rebased on commit 9fe8895a27a84 ("perf env: Add perf_env__cpuid, perf_env__{nr_}pmu_mappings")]
---
 tools/perf/util/evsel.c | 43 ++++++++++++++++++++++++++++++-----------
 1 file changed, 32 insertions(+), 11 deletions(-)
tools/perf/util/evsel.c +32 -11
index e8ff7a4bd490..03f720a85764 100644
@@ -2858,6 +2858,16 @@ static bool is_amd_ibs(struct evsel *evsel)
 	    || (evsel->pmu_name && !strncmp(evsel->pmu_name, "ibs", 3));
 }
 
+static bool is_amd_brs_event(struct evsel *evsel)
+{
+	return (evsel->core.attr.config & 0xff) == 0xc4;
+}
+
+static bool has_amd_brs(struct evsel *evsel)
+{
+	return (evsel->core.attr.sample_type & PERF_SAMPLE_BRANCH_STACK);
+}
+
 int evsel__open_strerror(struct evsel *evsel, struct target *target,
 			 int err, char *msg, size_t size)
 {
@@ -2947,6 +2957,28 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,
 	"We found oprofile daemon running, please stop it and try again.");
 		break;
 	case EINVAL:
+		if (is_amd(arch, cpuid)) {
+			if (is_amd_ibs(evsel)) {
+				if (evsel->core.attr.exclude_kernel)
+					return scnprintf(msg, size,
+	"AMD IBS can't exclude kernel events.  Try running at a higher privilege level.");
+				if (!evsel->core.system_wide)
+					return scnprintf(msg, size,
+	"AMD IBS may only be available in system-wide/per-cpu mode.  Try using -a, or -C and workload affinity");
+			}
+			if (has_amd_brs(evsel)) {
+				if (!is_amd_brs_event(evsel))
+					return scnprintf(msg, size,
+	"AMD Branch Sampling can only be used with the cpu/branch-brs,period=xxxx/ event or equivalent encodings.");
+				if (evsel->core.attr.freq)
+					return scnprintf(msg, size,
+	"AMD Branch Sampling does not support frequency mode sampling, must pass a fixed sampling period via -c option or cpu/branch-brs,period=xxxx/.");
+				/* another reason is that the period is too small */
+				return scnprintf(msg, size,
+	"AMD Branch Sampling does not support sampling period smaller than what is reported in /sys/devices/cpu/caps/branches.");
+			}
+		}
+
 		if (evsel->core.attr.sample_type & PERF_SAMPLE_CODE_PAGE_SIZE && perf_missing_features.code_page_size)
 			return scnprintf(msg, size, "Asking for the code page size isn't supported by this kernel.");
 		if (evsel->core.attr.sample_type & PERF_SAMPLE_DATA_PAGE_SIZE && perf_missing_features.data_page_size)
@@ -2963,17 +2995,6 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,
 			return scnprintf(msg, size,
 	"Invalid event (%s) in per-thread mode, enable system wide with '-a'.",
 					evsel__name(evsel));
-		if (is_amd(arch, cpuid)) {
-			if (is_amd_ibs(evsel)) {
-				if (evsel->core.attr.exclude_kernel)
-					return scnprintf(msg, size,
-	"AMD IBS can't exclude kernel events.  Try running at a higher privilege level.");
-				if (!evsel->core.system_wide)
-					return scnprintf(msg, size,
-	"AMD IBS may only be available in system-wide/per-cpu mode.  Try using -a, or -C and workload affinity");
-			}
-		}
-
 		break;
 	case ENODATA:
 		return scnprintf(msg, size, "Cannot collect data source with the load latency event alone. "
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
#27 lore raw

PATCH v7 13/13 perf report: add addr_from/addr_to sort dimensions

In reply to Stephane Eranian

With the existing symbol_from/symbol_to, branches captured in the same function would be collapsed into a single function if the latencies associated with the each branch (cycles) were all the same. That is the case on Intel Broadwell, for instance. Since Intel Skylake, the latency is captured by hardware and therefore is used to disambiguate branches.

Add addr_from/addr_to sort dimensions to sort branches based on their addresses and not the function there are in. The output is still the function name but the offset within the function is provided to uniquely identify each branch. These new sort dimensions also help with annotate because they create different entries in the histogram which, in turn, generates proper branch annotations.

Here is an example using AMD's branch sampling:

$ perf record -a -b -c 1000037 -e cpu/branch-brs/ test_prg
$ perf report
Samples: 6M of event 'cpu/branch-brs/', Event count (approx.): 6901276
Overhead  Command          Source Shared Object  Source Symbol                                   Target Symbol                                   Basic Block Cycle
  99.65%  test_prg	   test_prg              [.] test_thread                                 [.] test_thread                                 -
   0.02%  test_prg         [kernel.vmlinux]      [k] asm_sysvec_apic_timer_interrupt             [k] error_entry                                 -
$ perf report -F overhead,comm,dso,addr_from,addr_to
Samples: 6M of event 'cpu/branch-brs/', Event count (approx.): 6901276
Overhead  Command          Shared Object     Source Address          Target Address
   4.22%  test_prg         test_prg          [.] test_thread+0x3c    [.] test_thread+0x4
   4.13%  test_prg         test_prg          [.] test_thread+0x4     [.] test_thread+0x3a
   4.09%  test_prg         test_prg          [.] test_thread+0x3a    [.] test_thread+0x6
   4.08%  test_prg         test_prg          [.] test_thread+0x2     [.] test_thread+0x3c
   4.06%  test_prg         test_prg          [.] test_thread+0x3e    [.] test_thread+0x2
   3.87%  test_prg         test_prg          [.] test_thread+0x6     [.] test_thread+0x38
   3.84%  test_prg         test_prg          [.] test_thread         [.] test_thread+0x3e
   3.76%  test_prg         test_prg          [.] test_thread+0x1e    [.] test_thread
   3.76%  test_prg         test_prg          [.] test_thread+0x38    [.] test_thread+0x8
   3.56%  test_prg         test_prg          [.] test_thread+0x22    [.] test_thread+0x1e
   3.54%  test_prg         test_prg          [.] test_thread+0x8     [.] test_thread+0x36
   3.47%  test_prg         test_prg          [.] test_thread+0x1c    [.] test_thread+0x22
   3.45%  test_prg         test_prg          [.] test_thread+0x36    [.] test_thread+0xa
   3.28%  test_prg         test_prg          [.] test_thread+0x24    [.] test_thread+0x1c
   3.25%  test_prg         test_prg          [.] test_thread+0xa     [.] test_thread+0x34
   3.24%  test_prg         test_prg          [.] test_thread+0x1a    [.] test_thread+0x24
   3.20%  test_prg         test_prg          [.] test_thread+0x34    [.] test_thread+0xc
   3.04%  test_prg         test_prg          [.] test_thread+0x26    [.] test_thread+0x1a
   3.01%  test_prg         test_prg          [.] test_thread+0xc     [.] test_thread+0x32
   2.98%  test_prg         test_prg          [.] test_thread+0x18    [.] test_thread+0x26
   2.94%  test_prg         test_prg          [.] test_thread+0x32    [.] test_thread+0xe
   2.76%  test_prg         test_prg          [.] test_thread+0x28    [.] test_thread+0x18
   2.73%  test_prg         test_prg          [.] test_thread+0xe     [.] test_thread+0x30
   2.67%  test_prg         test_prg          [.] test_thread+0x30    [.] test_thread+0x10
   2.67%  test_prg         test_prg          [.] test_thread+0x16    [.] test_thread+0x28
   2.46%  test_prg         test_prg          [.] test_thread+0x10    [.] test_thread+0x2e
   2.44%  test_prg         test_prg          [.] test_thread+0x2a    [.] test_thread+0x16
   2.38%  test_prg         test_prg          [.] test_thread+0x14    [.] test_thread+0x2a
   2.32%  test_prg         test_prg          [.] test_thread+0x2e    [.] test_thread+0x12
   2.28%  test_prg         test_prg          [.] test_thread+0x12    [.] test_thread+0x2c
   2.16%  test_prg         test_prg          [.] test_thread+0x2c    [.] test_thread+0x14
   0.02%  test_prg         [kernel.vmlinux]  [k] asm_sysvec_apic_ti+0x5  [k] error_entry
---
 tools/perf/util/hist.c |   2 +
 tools/perf/util/hist.h |   2 +
 tools/perf/util/sort.c | 128 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/sort.h |   2 +
 4 files changed, 134 insertions(+)
tools/perf/util/hist.c +2 -0
index 0a8033b09e28..1c085ab56534 100644
@@ -124,6 +124,7 @@ void hists__calc_col_len(struct hists *hists, struct hist_entry *h)
 		} else {
 			symlen = unresolved_col_width + 4 + 2;
 			hists__new_col_len(hists, HISTC_SYMBOL_FROM, symlen);
+			hists__new_col_len(hists, HISTC_ADDR_FROM, symlen);
 			hists__set_unres_dso_col_len(hists, HISTC_DSO_FROM);
 		}
 
@@ -138,6 +139,7 @@ void hists__calc_col_len(struct hists *hists, struct hist_entry *h)
 		} else {
 			symlen = unresolved_col_width + 4 + 2;
 			hists__new_col_len(hists, HISTC_SYMBOL_TO, symlen);
+			hists__new_col_len(hists, HISTC_ADDR_TO, symlen);
 			hists__set_unres_dso_col_len(hists, HISTC_DSO_TO);
 		}
 
tools/perf/util/hist.h +2 -0
index 2a15e22fb89c..7ed4648d2fc2 100644
@@ -77,6 +77,8 @@ enum hist_column {
 	HISTC_GLOBAL_INS_LAT,
 	HISTC_LOCAL_P_STAGE_CYC,
 	HISTC_GLOBAL_P_STAGE_CYC,
+	HISTC_ADDR_FROM,
+	HISTC_ADDR_TO,
 	HISTC_NR_COLS, /* Last entry */
 };
 
tools/perf/util/sort.c +128 -0

146 lines not on this page · raw message

tools/perf/util/sort.h +2 -0
index f994261888e1..2ddc00d1c464 100644
@@ -251,6 +251,8 @@ enum sort_type {
 	SORT_SRCLINE_FROM,
 	SORT_SRCLINE_TO,
 	SORT_SYM_IPC,
+	SORT_ADDR_FROM,
+	SORT_ADDR_TO,
 
 	/* memory mode specific sort keys */
 	__SORT_MEMORY_MODE,
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
maintainer
#28 lore raw
In reply to Stephane Eranian
Quoted 6 lines · Stephane Eranian

V7 makes the following changes:
- rebased to 5.17-rc8
- Restrict BRS to be used only with RETIRED_TAKEN_BRANCH event in order to avoid adding more skid to other events in case that are monitored concurrently
- fixed bugs in the perf tool enhanced error handling for IBS and BRS

But afaict you failed to pick up the build fixes I did in my tree :/ Let me try and be careful preserving those while munging this in.

#29 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 6 lines · Stephane Eranian

V7 makes the following changes:
- rebased to 5.17-rc8
- Restrict BRS to be used only with RETIRED_TAKEN_BRANCH event in order to avoid adding more skid to other events in case that are monitored concurrently
- fixed bugs in the perf tool enhanced error handling for IBS and BRS

But afaict you failed to pick up the build fixes I did in my tree :/ Let me try and be careful preserving those while munging this in.

Sorry. I missed that. I can resubmit if you prefer.

maintainer
#30 lore raw
Quoted 10 lines · Stephane Eranian
Quoted 9 lines · Peter Zijlstra
Quoted 6 lines · Stephane Eranian

V7 makes the following changes:
- rebased to 5.17-rc8
- Restrict BRS to be used only with RETIRED_TAKEN_BRANCH event in order to avoid adding more skid to other events in case that are monitored concurrently
- fixed bugs in the perf tool enhanced error handling for IBS and BRS

Quoted 4 lines
Quoted 2 lines

But afaict you failed to pick up the build fixes I did in my tree :/ Let me try and be careful preserving those while munging this in.

Sorry. I missed that. I can resubmit if you prefer.

Nah, fixed it up again.. Even converted to static_call() this time.

Could you please test this branch?

https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=perf/amd

I've since done more Changelog edits, and I should do a comment with the static_call_mod() usage, but I think this ought to work.

At least it now actually builds...

#31 lore raw
Quoted 12 lines · Peter Zijlstra
Quoted 10 lines · Stephane Eranian
Quoted 9 lines · Peter Zijlstra
Quoted 6 lines · Stephane Eranian

V7 makes the following changes:
- rebased to 5.17-rc8
- Restrict BRS to be used only with RETIRED_TAKEN_BRANCH event in order to avoid adding more skid to other events in case that are monitored concurrently
- fixed bugs in the perf tool enhanced error handling for IBS and BRS

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

But afaict you failed to pick up the build fixes I did in my tree :/ Let me try and be careful preserving those while munging this in.

Sorry. I missed that. I can resubmit if you prefer.

Nah, fixed it up again.. Even converted to static_call() this time.

Could you please test this branch?

Started testing. BRS does not come up so far. Let me debug this.

Quoted 7 lines

https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=perf/amd

I've since done more Changelog edits, and I should do a comment with the static_call_mod() usage, but I think this ought to work.

At least it now actually builds...

#32 lore raw
Quoted 17 lines · Stephane Eranian
Quoted 15 lines · Peter Zijlstra
Quoted 13 lines · Stephane Eranian
Quoted 11 lines · Peter Zijlstra
Quoted 6 lines · Stephane Eranian

V7 makes the following changes:
- rebased to 5.17-rc8
- Restrict BRS to be used only with RETIRED_TAKEN_BRANCH event in order to avoid adding more skid to other events in case that are monitored concurrently
- fixed bugs in the perf tool enhanced error handling for IBS and BRS

But afaict you failed to pick up the build fixes I did in my tree :/ Let me try and be careful preserving those while munging this in.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Sorry. I missed that. I can resubmit if you prefer.

Nah, fixed it up again.. Even converted to static_call() this time.

Could you please test this branch?

Started testing. BRS does not come up so far. Let me debug this.

False alarm. I had not forced the config option. With the config option enabled, it works as expected.
I just found a minor issue in case BRS is not enabled. To be consistent, we should do:

arch/x86/events/perf_event.h +1 -1
@@ -1265,7 +1265,7 @@ static inline s64 amd_brs_adjust_period(s64 period)
 #else
 static inline int amd_brs_init(void)
 {
-       return 0;
+       return -EOPNOTSUPP;
 }
 

Thanks.

Quoted 7 lines
Quoted 7 lines

https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=perf/amd

I've since done more Changelog edits, and I should do a comment with the static_call_mod() usage, but I think this ought to work.

At least it now actually builds...

maintainer
#33 lore raw
Quoted 2 lines · Stephane Eranian

False alarm. I had not forced the config option. With the config option enabled, it works as expected.

Great!

arch/x86/events/perf_event.h static inline s64 amd_brs_adjust_period(s64 period) · line 1269

Quoted 12 lines

I just found a minor issue in case BRS is not enabled. To be consistent, we should do:

arch/x86/events/perf_event.h +1 -1
@@ -1265,7 +1265,7 @@ static inline s64 amd_brs_adjust_period(s64 period)
 #else
 static inline int amd_brs_init(void)
 {
-       return 0;
+       return -EOPNOTSUPP;
 }

Done!