perf/x86/amd: Add AMD Fam19h Branch Sampling support
Subsystems (3)
v5 cover perf/x86/amd: Add AMD Fam19h Branch Sampling support
This patch series adds support for the AMD Fam19h 16-deep branch sampling feature as described in the AMD PPR Fam19h Model 01h Revision B1 section 2.1.13. This is a model specific extension. It is not an architected AMD feature.
The Branch Sampling Feature (BRS) provides the statistical taken branch information necessary to enable autoFDO-style optimization by compilers, i.e., basic block execution counts.
BRS operates with a 16-deep saturating buffer in MSR registers. There is no hardware branch type filtering. All control flow changes are captured. BRS relies on specific programming of the core PMU of Fam19h. In particular, the following requirements must be met:
- the sampling period be greater than 16 (BRS depth)
- the sampling period must use fixed and not frequency mode
BRS interacts with the NMI interrupt as well. Because enabling BRS is expensive, it is only activated after P event occurrences, where P is the desired sampling period. At P occurrences of the event, the counter overflows, the CPU catches the NMI interrupt, activates BRS for 16 branches until it saturates, and then delivers the NMI to the kernel. Between the overflow and the time BRS activates more branches may be executed skewing the period. All along, the sampling event keeps counting. The skid may be attenuated by reducing the sampling period by 16.
BRS is integrated into perf_events seamlessly via the same PERF_RECORD_BRANCH_STACK sample format. BRS generates branch perf_branch_entry records in the sampling buffer. There is no prediction or latency information supported. The branches are stored in reverse order of execution. The most recent branch is the first entry in each record.
Because BRS must be stopped when a CPU goes into low power mode, the series includes patches to add callbacks on ACPI low power entry and exit which is used on AMD processors.
Given that there is no privilege filterting with BRS, the kernel implements filtering on privlege level.
This version adds a few simple modifications to perf record and report.
1. add the branch-brs event as a builtin such as it can used directly:
perf record -e branch-brs ...
2. improve error handling for AMD IBS and is contributed by Kim Phillips.
3. use the better error handling to improve error handling for BRS.
4. add two new sort dimensions to help display the branch sampling information. Because there is no latency information associated with the branch sampling feature perf report would collapse all samples within a function into a single histogram entry. This is expected because the default sort mode for PERF_SAMPLE_BRANCH_STACK is symbol_from/symbol_to. This propagates to the annotation.
For more detailed view of the branch samples, the new sort dimensions addr_from,addr_to can be used instead as follows:
$ perf report --sort=overhead,comm,dso,addr_from,addr_to
# Overhead Command Shared Object Source Address Target Address
# ........ .......... .............. .............. ..............
#
4.21% test_prg test_prg [.] test_threa+0x3c [.] test_threa+0x4
4.14% test_prg test_prg [.] test_threa+0x3e [.] test_threa+0x2
4.10% test_prg test_prg [.] test_threa+0x4 [.] test_threa+0x3a
4.07% test_prg test_prg [.] test_threa+0x2 [.] test_threa+0x3cVersus the default output:
$ perf report
# Overhead Command Source Shared Object Source Symbol Target Symbol Basic Block Cycles
# ........ ......... .................... ................ ................. ..................
#
99.52% test_prg test_prg [.] test_thread [.] test_thread - BRS can be used with any sampling event. However, it is recommended to use the RETIRED_BRANCH event because it matches what the BRS captures. For convenience, a pseudo event matching the branches captured by BRS is exported by the kernel (branch-brs):
$ perf record -b -e cpu/branch-brs/ -c 1000037 test
$ perf report -D 56531696056126 0x193c000 [0x1a8]: PERF_RECORD_SAMPLE(IP, 0x2): 18122/18230: 0x401d24 period: 1000037 addr: 0 ... branch stack: nr:16 ..... 0: 0000000000401d24 -> 0000000000401d5a 0 cycles 0 ..... 1: 0000000000401d5c -> 0000000000401d24 0 cycles 0 ..... 2: 0000000000401d22 -> 0000000000401d5c 0 cycles 0 ..... 3: 0000000000401d5e -> 0000000000401d22 0 cycles 0 ..... 4: 0000000000401d20 -> 0000000000401d5e 0 cycles 0 ..... 5: 0000000000401d3e -> 0000000000401d20 0 cycles 0 ..... 6: 0000000000401d42 -> 0000000000401d3e 0 cycles 0 ..... 7: 0000000000401d3c -> 0000000000401d42 0 cycles 0 ..... 8: 0000000000401d44 -> 0000000000401d3c 0 cycles 0 ..... 9: 0000000000401d3a -> 0000000000401d44 0 cycles 0 ..... 10: 0000000000401d46 -> 0000000000401d3a 0 cycles 0 ..... 11: 0000000000401d38 -> 0000000000401d46 0 cycles 0 ..... 12: 0000000000401d48 -> 0000000000401d38 0 cycles 0 ..... 13: 0000000000401d36 -> 0000000000401d48 0 cycles 0 ..... 14: 0000000000401d4a -> 0000000000401d36 0 cycles 0 ..... 15: 0000000000401d34 -> 0000000000401d4a 0 cycles 0 ... thread: test:18230 ...... dso: test
Special thanks to Kim Phillips @ AMD for the testing, reviews and contributions.
V2 makes the following changes:
- the low power callback code has be reworked completly. It is not impacting the generic perf_events code anymore. This is all handled via x86 code and only for ACPI low power driver which seems to be the default on AMD. The change in acpi_pad.c and processor_idle.c has no impact on non x86 architectures, on Intel x86 or AMD without BRS, a jump label is used to void the code unless necessary
- BRS is an opt-in compile time option for the kernel
- branch_stack bit clearing helper is introduced
- As for the fact that BRS holds the NMI and that it may conflict with other sampling events and introduced skid, this is not really a problem because AMD PMI skid is already very large prompting special handling in amd_pmu_wait_on_overflow(), so adding a few cycles while the CPU executes at most 16 taken branches is not a problem.
V3 makes the following changes:
- simplifies the handling of BRS enable/disable to mimic the Intel LBR code path more closely. That removes some callbacks in generic x86 code
- add config option to compile BRS as an opt-in (off by default)
- updated perf tool error reporting patch updates by Kim Phillips
V4 makes the following changes:
- rebase to latest tip.git (commit 6f5ac142e5df)
- integrate Kim Phillips latest perf tool error handling patches
V5 makes the following changes:
- rebased to 5.17-rc1
- integrated feedback from PeterZ about AMD perf_events callbacks
- fix cpufeature macros name X86_FEATURE_BRS
- integrated all perf tool error handling from kim Phillips
Kim Phillips (2):
perf evsel: Make evsel__env always return a valid env perf tools: Improve IBS error handling
Stephane Eranian (11):
perf/core: add perf_clear_branch_entry_bitfields() helper x86/cpufeatures: add AMD Fam19h Branch Sampling feature perf/x86/amd: add AMD Fam19h Branch Sampling support perf/x86/amd: add branch-brs helper event for Fam19h BRS perf/x86/amd: enable branch sampling priv level filtering perf/x86/amd: add AMD branch sampling period adjustment perf/x86/amd: make Zen3 branch sampling opt-in
ACPI: add perf low power callback
perf/x86/amd: add idle hooks for branch sampling perf tools: Improve error handling of AMD Branch Sampling perf report: add addr_from/addr_to sort dimensions
arch/x86/events/Kconfig | 8 + arch/x86/events/amd/Makefile | 1 + arch/x86/events/amd/brs.c | 363 +++++++++++++++++++++++++++++ arch/x86/events/amd/core.c | 217 ++++++++++++++++- arch/x86/events/core.c | 17 +- arch/x86/events/intel/lbr.c | 36 ++- arch/x86/events/perf_event.h | 143 ++++++++++-- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/msr-index.h | 4 + arch/x86/include/asm/perf_event.h | 21 ++ drivers/acpi/acpi_pad.c | 6 + drivers/acpi/processor_idle.c | 5 + include/linux/perf_event.h | 22 ++ tools/perf/util/evsel.c | 40 +++- tools/perf/util/hist.c | 2 + tools/perf/util/hist.h | 2 + tools/perf/util/sort.c | 128 ++++++++++ tools/perf/util/sort.h | 2 + 18 files changed, 977 insertions(+), 41 deletions(-) create mode 100644 arch/x86/events/amd/brs.c
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
PATCH v5 1/13 perf/core: add perf_clear_branch_entry_bitfields() helper
To make it simpler to reset all the info fields on the perf_branch_entry, this patch adds a helper inline function. The goal is to centralize the initialization to avoid missing a field in case more are added.
--- arch/x86/events/intel/lbr.c | 36 +++++++++++++++++------------------- include/linux/perf_event.h | 16 ++++++++++++++++ 2 files changed, 33 insertions(+), 19 deletions(-)
arch/x86/events/intel/lbr.c +17 -19
@@ -769,6 +769,7 @@ void intel_pmu_lbr_disable_all(void)void intel_pmu_lbr_read_32 (struct cpu_hw_events *cpuc) {unsigned long mask = x86_pmu.lbr_nr -1 ; +struct perf_branch_entry *br = cpuc->lbr_entries;u64 tos =intel_pmu_lbr_tos ();int i;
@@ -784,15 +785,11 @@ void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)rdmsrl (x86_pmu.lbr_from + lbr_idx, msr_lastbranch.lbr); - cpuc->lbr_entries[i].from = msr_lastbranch.from; - cpuc->lbr_entries[i].to = msr_lastbranch.to; - cpuc->lbr_entries[i].mispred =0 ; - cpuc->lbr_entries[i].predicted =0 ; - cpuc->lbr_entries[i].in_tx =0 ; - cpuc->lbr_entries[i].abort =0 ; - cpuc->lbr_entries[i].cycles =0 ; - cpuc->lbr_entries[i].type =0 ; - cpuc->lbr_entries[i].reserved =0 ; +perf_clear_branch_entry_bitfields (br); + + br->from = msr_lastbranch.from; + br->to = msr_lastbranch.to; + br++; } cpuc->lbr_stack.nr = i; cpuc->lbr_stack.hw_idx = tos;
@@ -807,6 +804,7 @@ void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc) {bool need_info = false, call_stack = false;unsigned long mask = x86_pmu.lbr_nr -1 ; +struct perf_branch_entry *br = cpuc->lbr_entries;u64 tos =intel_pmu_lbr_tos ();int i;int out =0 ;
@@ -878,15 +876,14 @@ void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)if (abort && x86_pmu.lbr_double_abort && out >0 ) out--; - cpuc->lbr_entries[out].from = from; - cpuc->lbr_entries[out].to = to; - cpuc->lbr_entries[out].mispred = mis; - cpuc->lbr_entries[out].predicted = pred; - cpuc->lbr_entries[out].in_tx = in_tx; - cpuc->lbr_entries[out].abort = abort; - cpuc->lbr_entries[out].cycles = cycles; - cpuc->lbr_entries[out].type =0 ; - cpuc->lbr_entries[out].reserved =0 ; +perf_clear_branch_entry_bitfields (br+out); + br[out].from = from; + br[out].to = to; + br[out].mispred = mis; + br[out].predicted = pred; + br[out].in_tx = in_tx; + br[out].abort = abort; + br[out].cycles = cycles; out++; } cpuc->lbr_stack.nr = out;
@@ -951,6 +948,8 @@ static void intel_pmu_store_lbr(struct cpu_hw_events *cpuc, to =rdlbr_to (i, lbr); info =rdlbr_info (i, lbr); +perf_clear_branch_entry_bitfields (e); + e->from = from; e->to = to; e->mispred =get_lbr_mispred (info);
@@ -959,7 +958,6 @@ static void intel_pmu_store_lbr(struct cpu_hw_events *cpuc, e->abort = !!(info &LBR_INFO_ABORT ); e->cycles =get_lbr_cycles (info); e->type =get_lbr_br_type (info); - e->reserved =0 ; } cpuc->lbr_stack.nr = i;
include/linux/perf_event.h +16 -0
@@ -1063,6 +1063,22 @@ static inline void perf_sample_data_init(struct perf_sample_data *data, data->txn =0 ; } +/* +* Clear all bitfields in the perf_branch_entry. +* The to and from fields are not cleared because they are +* systematically modified by caller. +*/ +static inline void perf_clear_branch_entry_bitfields (struct perf_branch_entry *br) +{ + br->mispred =0 ; + br->predicted =0 ; + br->in_tx =0 ; + br->abort =0 ; + br->cycles =0 ; + br->type =0 ; + br->reserved =0 ; +} + extern void perf_output_sample(struct perf_output_handle *handle, struct perf_event_header *header, struct perf_sample_data *data,
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
PATCH v5 2/13 x86/cpufeatures: add AMD Fam19h Branch Sampling feature
In reply to Stephane EranianThis patch adds a cpu feature for AMD Fam19h Branch Sampling feature as bit 31 of EBX on CPUID leaf function 0x80000008.
--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -318,6 +318,7 @@#define X86_FEATURE_VIRT_SSBD (13*32+25)/* Virtualized Speculative Store Bypass Disable */ #define X86_FEATURE_AMD_SSB_NO (13*32+26)/* "" Speculative Store Bypass is fixed in hardware. */ #define X86_FEATURE_CPPC (13*32+27)/* Collaborative Processor Performance Control */ +#define X86_FEATURE_BRS (13*32+31)/* Branch Sampling available */ /* Thermal and Power Management Leaf, CPUID level 0x00000006 (EAX), word 14 */ #define X86_FEATURE_DTHERM (14*32+ 0)/* Digital Thermal Sensor */
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
PATCH v5 3/13 perf/x86/amd: add AMD Fam19h Branch Sampling support
In reply to Stephane EranianThis patch adds support for the AMD Fam19h 16-deep branch sampling feature as described in the AMD PPR Fam19h Model 01h Revision B1. This is a model specific extension. It is not an architected AMD feature.
The Branch Sampling (BRS) operates with a 16-deep saturating buffer in MSR registers. There is no branch type filtering. All control flow changes are captured. BRS relies on specific programming of the core PMU of Fam19h. In particular, the following requirements must be met:
- the sampling period be greater than 16 (BRS depth)
- the sampling period must use a fixed and not frequency mode
BRS interacts with the NMI interrupt as well. Because enabling BRS is expensive, it is only activated after P event occurrences, where P is the desired sampling period. At P occurrences of the event, the counter overflows, the CPU catches the interrupt, activates BRS for 16 branches until it saturates, and then delivers the NMI to the kernel. Between the overflow and the time BRS activates more branches may be executed skewing the period. All along, the sampling event keeps counting. The skid may be attenuated by reducing the sampling period by 16 (subsequent patch).
BRS is integrated into perf_events seamlessly via the same PERF_RECORD_BRANCH_STACK sample format. BRS generates perf_branch_entry records in the sampling buffer. No prediction information is supported. The branches are stored in reverse order of execution. The most recent branch is the first entry in each record.
No modification to the perf tool is necessary.
BRS can be used with any sampling event. However, it is recommended to use the RETIRED_BRANCH_INSTRUCTIONS event because it matches what the BRS captures.
$ perf record -b -c 1000037 -e cpu/event=0xc2,name=ret_br_instructions/ test
$ perf report -D 56531696056126 0x193c000 [0x1a8]: PERF_RECORD_SAMPLE(IP, 0x2): 18122/18230: 0x401d24 period: 1000037 addr: 0 ... branch stack: nr:16 ..... 0: 0000000000401d24 -> 0000000000401d5a 0 cycles 0 ..... 1: 0000000000401d5c -> 0000000000401d24 0 cycles 0 ..... 2: 0000000000401d22 -> 0000000000401d5c 0 cycles 0 ..... 3: 0000000000401d5e -> 0000000000401d22 0 cycles 0 ..... 4: 0000000000401d20 -> 0000000000401d5e 0 cycles 0 ..... 5: 0000000000401d3e -> 0000000000401d20 0 cycles 0 ..... 6: 0000000000401d42 -> 0000000000401d3e 0 cycles 0 ..... 7: 0000000000401d3c -> 0000000000401d42 0 cycles 0 ..... 8: 0000000000401d44 -> 0000000000401d3c 0 cycles 0 ..... 9: 0000000000401d3a -> 0000000000401d44 0 cycles 0 ..... 10: 0000000000401d46 -> 0000000000401d3a 0 cycles 0 ..... 11: 0000000000401d38 -> 0000000000401d46 0 cycles 0 ..... 12: 0000000000401d48 -> 0000000000401d38 0 cycles 0 ..... 13: 0000000000401d36 -> 0000000000401d48 0 cycles 0 ..... 14: 0000000000401d4a -> 0000000000401d36 0 cycles 0 ..... 15: 0000000000401d34 -> 0000000000401d4a 0 cycles 0 ... thread: test:18230 ...... dso: test
--- arch/x86/events/amd/Makefile | 2 +- arch/x86/events/amd/brs.c | 317 +++++++++++++++++++++++++++++++ arch/x86/events/amd/core.c | 197 ++++++++++++++++++- arch/x86/events/core.c | 10 +- arch/x86/events/perf_event.h | 101 ++++++++-- arch/x86/include/asm/msr-index.h | 4 + 6 files changed, 609 insertions(+), 22 deletions(-) create mode 100644 arch/x86/events/amd/brs.c
arch/x86/events/amd/Makefile +1 -1
@@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_CPU_SUP_AMD) += core.o +obj-$(CONFIG_CPU_SUP_AMD) += core.o brs.o obj-$(CONFIG_PERF_EVENTS_AMD_POWER) += power.o obj-$(CONFIG_X86_LOCAL_APIC) += ibs.o obj-$(CONFIG_PERF_EVENTS_AMD_UNCORE) += amd-uncore.o
arch/x86/events/amd/brs.c new file +317 -0
317 lines not on this page · raw message
arch/x86/events/amd/core.c +193 -4
284 lines not on this page · raw message
arch/x86/events/core.c +9 -1
@@ -1338,6 +1338,10 @@ static void x86_pmu_enable(struct pmu *pmu)if (hwc->state &PERF_HES_ARCH )continue ; +/* +* if cpuc->enabled = 0, then no wrmsr as +* per x86_pmu_enable_event() +*/ x86_pmu_start (event,PERF_EF_RELOAD ); } cpuc->n_added =0 ;
@@ -1704,11 +1708,15 @@ int x86_pmu_handle_irq(struct pt_regs *regs)* event overflow */ handled++; -perf_sample_data_init (&data,0 , event->hw.last_period);if (!x86_perf_event_set_period (event))continue ; +perf_sample_data_init (&data,0 , event->hw.last_period); + +if (has_branch_stack (event)) + data.br_stack = &cpuc->lbr_stack; +if (perf_event_overflow (event, &data, regs))x86_pmu_stop (event,0 ); }
arch/x86/events/perf_event.h +85 -16
131 lines not on this page · raw message
arch/x86/include/asm/msr-index.h +4 -0
@@ -667,6 +667,10 @@#define MSR_IA32_PERF_CTL 0x00000199#define INTEL_PERF_CTL_MASK 0xffff +/* AMD Branch Sampling configuration */ +#define MSR_AMD_DBG_EXTN_CFG 0xc000010f +#define MSR_AMD_SAMP_BR_FROM 0xc0010300 +#define MSR_IA32_MPERF 0x000000e7#define MSR_IA32_APERF 0x000000e8
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
kernel test robot
build botRe: [PATCH v5 03/13] perf/x86/amd: add AMD Fam19h Branch Sampling support
Thank you for the patch! Perhaps something to improve:
Hi Stephane,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on next-20220125]
[cannot apply to tip/x86/core rafael-pm/linux-next tip/perf/core v5.17-rc1 v5.16 v5.16-rc8 v5.17-rc1] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Stephane-Eranian/perf-core-add-perf_clear_branch_entry_bitfields-helper/20220127-083516 base: d25ee88530253138d0b20d43511ca5acbda4e9f7 config: i386-buildonly-randconfig-r002-20220124 (https://download.01.org/0day-ci/archive/20220127/202201270923.FG90uMou-lkp(a)intel.com/config) compiler: gcc-9 (Debian 9.3.0-22) 9.3.0 reproduce (this is a W=1 build): # https://github.com/0day-ci/linux/commit/933d072f6e0c0409115a8038a89a56979a042d30 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Stephane-Eranian/perf-core-add-perf_clear_branch_entry_bitfields-helper/20220127-083516 git checkout 933d072f6e0c0409115a8038a89a56979a042d30 # save the config file to linux build tree mkdir build_dir make W=1 O=build_dir ARCH=i386 SHELL=/bin/bash arch/x86/events/amd/
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
arch/x86/events/amd/core.c: In function 'amd_pmu_enable_all':
>> arch/x86/events/amd/core.c:676:24: warning: variable 'hwc' set but not used [-Wunused-but-set-variable]
676 | struct hw_perf_event *hwc;
| ^~~vim +/hwc +676 arch/x86/events/amd/core.c
672
673 static void amd_pmu_enable_all(int added)
674 {
675 struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
> 676 struct hw_perf_event *hwc;
677 int idx;
678
679 amd_brs_enable_all();
680
681 for (idx = 0; idx < x86_pmu.num_counters; idx++) {
682 hwc = &cpuc->events[idx]->hw;
683
684 /* only activate events which are marked as active */
685 if (!test_bit(idx, cpuc->active_mask))
686 continue;
687
688 amd_pmu_enable_event(cpuc->events[idx]);
689 }
690 }
691
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kernel test robot
build botRe: [PATCH v5 03/13] perf/x86/amd: add AMD Fam19h Branch Sampling support
In reply to Stephane EranianThank you for the patch! Perhaps something to improve:
Hi Stephane,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on next-20220125]
[cannot apply to tip/x86/core rafael-pm/linux-next tip/perf/core v5.17-rc1 v5.16 v5.16-rc8 v5.17-rc1] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Stephane-Eranian/perf-core-add-perf_clear_branch_entry_bitfields-helper/20220127-083516 base: d25ee88530253138d0b20d43511ca5acbda4e9f7 config: i386-randconfig-a011-20220124 (https://download.01.org/0day-ci/archive/20220127/202201271410.cro07bYq-lkp(a)intel.com/config) compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project f400a6012c668dfaa73462caf067ceb074e66c47)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/933d072f6e0c0409115a8038a89a56979a042d30
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Stephane-Eranian/perf-core-add-perf_clear_branch_entry_bitfields-helper/20220127-083516 git checkout 933d072f6e0c0409115a8038a89a56979a042d30
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash arch/x86/events/amd/ drivers/acpi/
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
>> arch/x86/events/amd/core.c:676:24: warning: variable 'hwc' set but not used [-Wunused-but-set-variable]
struct hw_perf_event *hwc;
^1 warning generated.
vim +/hwc +676 arch/x86/events/amd/core.c
672
673 static void amd_pmu_enable_all(int added)
674 {
675 struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
> 676 struct hw_perf_event *hwc;
677 int idx;
678
679 amd_brs_enable_all();
680
681 for (idx = 0; idx < x86_pmu.num_counters; idx++) {
682 hwc = &cpuc->events[idx]->hw;
683
684 /* only activate events which are marked as active */
685 if (!test_bit(idx, cpuc->active_mask))
686 continue;
687
688 amd_pmu_enable_event(cpuc->events[idx]);
689 }
690 }
691
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
PATCH v5 4/13 perf/x86/amd: add branch-brs helper event for Fam19h BRS
In reply to Stephane EranianThis patch adds a pseudo event called branch-brs to help use the FAM Fam19h Branch Sampling feature (BRS). BRS samples taken branches, so it is best used when sampling on a retired taken branch event (0xc4) which is what BRS captures. Instead of trying to remember the event code or actual event name, users can simply do:
$ perf record -b -e cpu/branch-brs/ -c 1000037 .....
--- arch/x86/events/amd/core.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arch/x86/events/amd/core.c +16 -0
@@ -1105,8 +1105,24 @@ static struct attribute_group group_caps_amd_brs = { .is_visible = amd_brs_is_visible, }; +#define AMD_FAM19H_BRS_EVENT 0xc4/* Fam19h RETIRED_TAKEN_BRANCH_INSTRUCTIONS */ +EVENT_ATTR_STR (branch-brs, amd_branch_brs, + "event=" __stringify(AMD_FAM19H_BRS_EVENT)"\n"); + +static struct attribute *amd_brs_events_attrs[] = { +EVENT_PTR (amd_branch_brs), +NULL , +}; + +static struct attribute_group group_events_amd_brs = { + .name ="events" , + .attrs = amd_brs_events_attrs, + .is_visible = amd_brs_is_visible, +}; +static const struct attribute_group *amd_attr_update[] = { &group_caps_amd_brs, + &group_events_amd_brs,NULL , };
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
PATCH v5 5/13 perf/x86/amd: enable branch sampling priv level filtering
In reply to Stephane EranianThe AMD Branch Sampling features does not provide hardware filtering by privilege level. The associated PMU counter does but not the branch sampling by itself. Given how BRS operates there is a possibility that BRS captures kernel level branches even though the event is programmed to count only at the user level. This patch implements a workaround in software by removing the branches which belong to the wrong privilege level. The privilege level is evaluated on the target of the branch and not the source so as to be compatible with other architectures. As a consequence of this patch, the number of entries in the PERF_RECORD_BRANCH_STACK buffer may be less than the maximum (16). It could even be zero. Another consequence is that consecutive entries in the branch stack may not reflect actual code path and may have discontinuities, in case kernel branches were suppressed. But this is no different than what happens on other architectures.
--- arch/x86/events/amd/brs.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)
arch/x86/events/amd/brs.c +20 -6
@@ -92,10 +92,6 @@ int amd_brs_setup_filter(struct perf_event *event)if ((type & ~PERF_SAMPLE_BRANCH_PLM_ALL ) !=PERF_SAMPLE_BRANCH_ANY )return -EINVAL ; -/* can only capture at all priv levels due to the way BRS works */ -if ((type &PERF_SAMPLE_BRANCH_PLM_ALL ) !=PERF_SAMPLE_BRANCH_PLM_ALL ) -return -EINVAL ; -return 0 ; }
@@ -195,6 +191,21 @@ void amd_brs_disable_all(void)amd_brs_disable (); } +static bool amd_brs_match_plm (struct perf_event *event,u64 to) +{ +int type = event->attr.branch_sample_type; +int plm_k =PERF_SAMPLE_BRANCH_KERNEL |PERF_SAMPLE_BRANCH_HV ; +int plm_u =PERF_SAMPLE_BRANCH_USER ; + +if (!(type & plm_k) &&kernel_ip (to)) +return 0 ; + +if (!(type & plm_u) && !kernel_ip (to)) +return 0 ; + +return 1 ; +} +/* * Caller must ensure amd_brs_inuse() is true before calling * return:
@@ -252,8 +263,6 @@ void amd_brs_drain(void)if (to ==BRS_POISON )break ; -rdmsrl (brs_from (brs_idx), from); -/* * Sign-extend SAMP_BR_TO to 64 bits, bits 61-63 are reserved. * Necessary to generate proper virtual addresses suitable for
@@ -261,6 +270,11 @@ void amd_brs_drain(void)*/ to = (u64)(((s64 )to << shift) >> shift); +if (!amd_brs_match_plm (event, to)) +continue ; + +rdmsrl (brs_from (brs_idx), from); +perf_clear_branch_entry_bitfields (br+nr); br[nr].from = from;
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
PATCH v5 6/13 perf/x86/amd: add AMD branch sampling period adjustment
In reply to Stephane EranianThis supplemental patch adds the code to adjust the sampling event period when used with the Branch Sampling feature (BRS). Given the depth of the BRS (16), the period is reduced by that depth such that in the best case scenario, BRS saturates at the desired sampling period. In practice, though, the processor may execute more branches. Given a desired period P and a depth D, the kernel programs the actual period at P - D. After P occurrences of the sampling event, the counter overflows. It then may take X branches (skid) before the NMI is caught and held by the hardware and BRS activates. Then, after D branches, BRS saturates and the NMI is delivered. With no skid, the effective period would be (P - D) + D = P. In practice, however, it will likely be (P - D) + X + D. There is no way to eliminate X or predict X.
--- arch/x86/events/core.c | 7 +++++++ arch/x86/events/perf_event.h | 12 ++++++++++++ 2 files changed, 19 insertions(+)
arch/x86/events/core.c +7 -0
@@ -1374,6 +1374,13 @@ int x86_perf_event_set_period(struct perf_event *event) x86_pmu.set_topdown_event_period)return x86_pmu.set_topdown_event_period (event); +/* +* decrease period by the depth of the BRS feature to get +* the last N taken branches and approximate the desired period +*/ +if (has_branch_stack (event)) + period =amd_brs_adjust_period (period); +/* * If we are way outside a reasonable range then just skip forward: */
arch/x86/events/perf_event.h +12 -0
@@ -1263,6 +1263,14 @@ static inline bool amd_brs_active(void) return cpuc->brs_active; } +static inline s64 amd_brs_adjust_period (s64 period) +{ +if (period > x86_pmu.lbr_nr) +return period - x86_pmu.lbr_nr; + +return period; +} +#else /* CONFIG_CPU_SUP_AMD */ static inline int amd_pmu_init (void )
@@ -1287,6 +1295,10 @@ static inline void amd_brs_disable_all(void) { } +static inline s64 amd_brs_adjust_period (s64 period) +{ +return period; +}#endif /* CONFIG_CPU_SUP_AMD */ static inline int is_pebs_pt (struct perf_event *event)
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
PATCH v5 7/13 perf/x86/amd: make Zen3 branch sampling opt-in
In reply to Stephane EranianThis patch adds a kernel config option to make support for AMD Zen3 Branch Sampling (BRS) an opt-in compile time option.
--- arch/x86/events/Kconfig | 8 ++++++ arch/x86/events/amd/Makefile | 3 ++- arch/x86/events/perf_event.h | 49 ++++++++++++++++++++++++++++-------- 3 files changed, 49 insertions(+), 11 deletions(-)
arch/x86/events/Kconfig +8 -0
@@ -44,4 +44,12 @@ config PERF_EVENTS_AMD_UNCORE To compile this driver as a module, choose M here: the module will be called 'amd-uncore'. + +config PERF_EVENTS_AMD_BRS + depends on PERF_EVENTS && CPU_SUP_AMD + tristate "AMD Zen3 Branch Sampling support" + help + Enable AMD Zen3 branch sampling support (BRS) which samples up to + 16 consecutive taken branches in registers. + endmenu
arch/x86/events/amd/Makefile +2 -1
@@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_CPU_SUP_AMD) += core.o brs.o +obj-$(CONFIG_CPU_SUP_AMD) += core.o +obj-$(CONFIG_PERF_EVENTS_AMD_BRS) += brs.o obj-$(CONFIG_PERF_EVENTS_AMD_POWER) += power.o obj-$(CONFIG_X86_LOCAL_APIC) += ibs.o obj-$(CONFIG_PERF_EVENTS_AMD_UNCORE) += amd-uncore.o
arch/x86/events/perf_event.h +39 -10
70 lines not on this page · raw message
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
Peter Zijlstra
maintainerRe: [PATCH v5 07/13] perf/x86/amd: make Zen3 branch sampling opt-in
Quoted 16 lines · Stephane Eranian
This patch adds a kernel config option to make support for AMD Zen3 Branch Sampling (BRS) an opt-in compile time option.
Signed-off-by Stephane Eranian <[email protected]>--- arch/x86/events/Kconfig | 8 ++++++ arch/x86/events/amd/Makefile | 3 ++- arch/x86/events/perf_event.h | 49 ++++++++++++++++++++++++++++-------- 3 files changed, 49 insertions(+), 11 deletions(-)arch/x86/events/Kconfig +0
-0
arch/x86/events/Kconfig config PERF_EVENTS_AMD_UNCORE · line 50
Quoted 7 lines
arch/x86/events/Kconfig +4
-0@@ -44,4 +44,12 @@ config PERF_EVENTS_AMD_UNCORE To compile this driver as a module, choose M here: the module will be called 'amd-uncore'. + +config PERF_EVENTS_AMD_BRS + depends on PERF_EVENTS && CPU_SUP_AMD + tristate "AMD Zen3 Branch Sampling support"
tristate ?!
Re: [PATCH v5 07/13] perf/x86/amd: make Zen3 branch sampling opt-in
Quoted 19 lines · Peter Zijlstra
Quoted 17 lines · Stephane Eranian
This patch adds a kernel config option to make support for AMD Zen3 Branch Sampling (BRS) an opt-in compile time option.
Signed-off-by Stephane Eranian <[email protected]>--- arch/x86/events/Kconfig | 8 ++++++ arch/x86/events/amd/Makefile | 3 ++- arch/x86/events/perf_event.h | 49 ++++++++++++++++++++++++++++-------- 3 files changed, 49 insertions(+), 11 deletions(-)arch/x86/events/Kconfig +0
-0@@ -44,4 +44,12 @@ config PERF_EVENTS_AMD_UNCORE
Quoted 8 lines
arch/x86/events/Kconfig config PERF_EVENTS_AMD_UNCORE · line 50
Quoted 6 lines
arch/x86/events/Kconfig +4
-0To compile this driver as a module, choose M here: the module will be called 'amd-uncore'. + +config PERF_EVENTS_AMD_BRS + depends on PERF_EVENTS && CPU_SUP_AMD + tristate "AMD Zen3 Branch Sampling support"tristate ?!
Ok, I will fix that in V6.
Thanks.
PATCH v5 8/13 ACPI: add perf low power callback
In reply to Stephane EranianThis patch add an optional callback needed by some PMU features, e.g., AMD BRS, to give a chance to the perf_events code to change its state before going to low power and after coming back.
The callback is void when the PERF_NEEDS_LOPWR_CB flag is not set. This flag must be set in arch specific perf_event.h header whenever needed. When not set, there is no impact of the ACPI code.
--- drivers/acpi/acpi_pad.c | 6 ++++++ drivers/acpi/processor_idle.c | 5 +++++ include/linux/perf_event.h | 6 ++++++ 3 files changed, 17 insertions(+)
drivers/acpi/acpi_pad.c +6 -0
@@ -164,6 +164,9 @@ static int power_saving_thread(void *data) tsc_marked_unstable =1 ; }local_irq_disable (); + +perf_lopwr_cb (true); +tick_broadcast_enable ();tick_broadcast_enter ();stop_critical_timings ();
@@ -172,6 +175,9 @@ static int power_saving_thread(void *data)start_critical_timings ();tick_broadcast_exit (); + +perf_lopwr_cb (false); +local_irq_enable ();if (time_before (expire_time, jiffies)) {
drivers/acpi/processor_idle.c +5 -0
@@ -21,6 +21,7 @@#include <linux/cpuidle.h> #include <linux/cpu.h> #include <linux/minmax.h> +#include <linux/perf_event.h> #include <acpi/processor.h> /*
@@ -544,6 +545,8 @@ static void wait_for_freeze(void)*/ static void __cpuidle acpi_idle_do_entry(struct acpi_processor_cx *cx) { +perf_lopwr_cb (true); +if (cx->entry_method ==ACPI_CSTATE_FFH ) {/* Call into architectural FFH based C-state */ acpi_processor_ffh_cstate_enter (cx);
@@ -554,6 +557,8 @@ static void __cpuidle acpi_idle_do_entry(struct acpi_processor_cx *cx)inb (cx->address);wait_for_freeze (); } + +perf_lopwr_cb (false); }/**
include/linux/perf_event.h +6 -0
@@ -1676,4 +1676,10 @@ typedef int (perf_snapshot_branch_stack_t)(struct perf_branch_entry *entries, unsigned int cnt);DECLARE_STATIC_CALL (perf_snapshot_branch_stack, perf_snapshot_branch_stack_t); +#ifndef PERF_NEEDS_LOPWR_CB +static inline void perf_lopwr_cb (bool mode) +{ +} +#endif +#endif /* _LINUX_PERF_EVENT_H */
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
kernel test robot
build botRe: [PATCH v5 08/13] ACPI: add perf low power callback
Thank you for the patch! Yet something to improve:
Hi Stephane,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on next-20220125]
[cannot apply to tip/x86/core rafael-pm/linux-next tip/perf/core v5.17-rc1 v5.16 v5.16-rc8 v5.17-rc1] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Stephane-Eranian/perf-core-add-perf_clear_branch_entry_bitfields-helper/20220127-083516 base: d25ee88530253138d0b20d43511ca5acbda4e9f7 config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20220127/202201271224.g4ALolMJ-lkp(a)intel.com/config) compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/fe7943af61c65b96cff36b8388246531402e7f26
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Stephane-Eranian/perf-core-add-perf_clear_branch_entry_bitfields-helper/20220127-083516 git checkout fe7943af61c65b96cff36b8388246531402e7f26
# save the config file to linux build tree
mkdir build_dir
make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash drivers/
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
drivers/acpi/acpi_pad.c: In function 'power_saving_thread':
>> drivers/acpi/acpi_pad.c:168:4: error: implicit declaration of function 'perf_lopwr_cb' [-Werror=implicit-function-declaration]
168 | perf_lopwr_cb(true);
| ^~~~~~~~~~~~~cc1: some warnings being treated as errors
vim +/perf_lopwr_cb +168 drivers/acpi/acpi_pad.c
135
136 static unsigned int idle_pct = 5; /* percentage */
137 static unsigned int round_robin_time = 1; /* second */
138 static int power_saving_thread(void *data)
139 {
140 int do_sleep;
141 unsigned int tsk_index = (unsigned long)data;
142 u64 last_jiffies = 0;
143
144 sched_set_fifo_low(current);
145
146 while (!kthread_should_stop()) {
147 unsigned long expire_time;
148
149 /* round robin to cpus */
150 expire_time = last_jiffies + round_robin_time * HZ;
151 if (time_before(expire_time, jiffies)) {
152 last_jiffies = jiffies;
153 round_robin_cpu(tsk_index);
154 }
155
156 do_sleep = 0;
157
158 expire_time = jiffies + HZ * (100 - idle_pct) / 100;
159
160 while (!need_resched()) {
161 if (tsc_detected_unstable && !tsc_marked_unstable) {
162 /* TSC could halt in idle, so notify users */
163 mark_tsc_unstable("TSC halts in idle");
164 tsc_marked_unstable = 1;
165 }
166 local_irq_disable();
167
> 168 perf_lopwr_cb(true);
169
170 tick_broadcast_enable();
171 tick_broadcast_enter();
172 stop_critical_timings();
173
174 mwait_idle_with_hints(power_saving_mwait_eax, 1);
175
176 start_critical_timings();
177 tick_broadcast_exit();
178
179 perf_lopwr_cb(false);
180
181 local_irq_enable();
182
183 if (time_before(expire_time, jiffies)) {
184 do_sleep = 1;
185 break;
186 }
187 }
188
189 /*
190 * current sched_rt has threshold for rt task running time.
191 * When a rt task uses 95% CPU time, the rt thread will be
192 * scheduled out for 5% CPU time to not starve other tasks. But
193 * the mechanism only works when all CPUs have RT task running,
194 * as if one CPU hasn't RT task, RT task from other CPUs will
195 * borrow CPU time from this CPU and cause RT task use > 95%
196 * CPU time. To make 'avoid starvation' work, takes a nap here.
197 */
198 if (unlikely(do_sleep))
199 schedule_timeout_killable(HZ * idle_pct / 100);
200
201 /* If an external event has set the need_resched flag, then
202 * we need to deal with it, or this loop will continue to
203 * spin without calling __mwait().
204 */
205 if (unlikely(need_resched()))
206 schedule();
207 }
208
209 exit_round_robin(tsk_index);
210 return 0;
211 }
212
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kernel test robot
build botRe: [PATCH v5 08/13] ACPI: add perf low power callback
In reply to Stephane EranianThank you for the patch! Yet something to improve:
Hi Stephane,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on next-20220125]
[cannot apply to tip/x86/core rafael-pm/linux-next tip/perf/core v5.17-rc1 v5.16 v5.16-rc8 v5.17-rc1] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Stephane-Eranian/perf-core-add-perf_clear_branch_entry_bitfields-helper/20220127-083516 base: d25ee88530253138d0b20d43511ca5acbda4e9f7 config: i386-randconfig-a011-20220124 (https://download.01.org/0day-ci/archive/20220127/202201271501.7zuL3YEw-lkp(a)intel.com/config) compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project f400a6012c668dfaa73462caf067ceb074e66c47)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/fe7943af61c65b96cff36b8388246531402e7f26
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Stephane-Eranian/perf-core-add-perf_clear_branch_entry_bitfields-helper/20220127-083516 git checkout fe7943af61c65b96cff36b8388246531402e7f26
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash drivers/acpi/
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
>> drivers/acpi/acpi_pad.c:168:4: error: implicit declaration of function 'perf_lopwr_cb' [-Werror,-Wimplicit-function-declaration]
perf_lopwr_cb(true);
^1 error generated.
vim +/perf_lopwr_cb +168 drivers/acpi/acpi_pad.c
135
136 static unsigned int idle_pct = 5; /* percentage */
137 static unsigned int round_robin_time = 1; /* second */
138 static int power_saving_thread(void *data)
139 {
140 int do_sleep;
141 unsigned int tsk_index = (unsigned long)data;
142 u64 last_jiffies = 0;
143
144 sched_set_fifo_low(current);
145
146 while (!kthread_should_stop()) {
147 unsigned long expire_time;
148
149 /* round robin to cpus */
150 expire_time = last_jiffies + round_robin_time * HZ;
151 if (time_before(expire_time, jiffies)) {
152 last_jiffies = jiffies;
153 round_robin_cpu(tsk_index);
154 }
155
156 do_sleep = 0;
157
158 expire_time = jiffies + HZ * (100 - idle_pct) / 100;
159
160 while (!need_resched()) {
161 if (tsc_detected_unstable && !tsc_marked_unstable) {
162 /* TSC could halt in idle, so notify users */
163 mark_tsc_unstable("TSC halts in idle");
164 tsc_marked_unstable = 1;
165 }
166 local_irq_disable();
167
> 168 perf_lopwr_cb(true);
169
170 tick_broadcast_enable();
171 tick_broadcast_enter();
172 stop_critical_timings();
173
174 mwait_idle_with_hints(power_saving_mwait_eax, 1);
175
176 start_critical_timings();
177 tick_broadcast_exit();
178
179 perf_lopwr_cb(false);
180
181 local_irq_enable();
182
183 if (time_before(expire_time, jiffies)) {
184 do_sleep = 1;
185 break;
186 }
187 }
188
189 /*
190 * current sched_rt has threshold for rt task running time.
191 * When a rt task uses 95% CPU time, the rt thread will be
192 * scheduled out for 5% CPU time to not starve other tasks. But
193 * the mechanism only works when all CPUs have RT task running,
194 * as if one CPU hasn't RT task, RT task from other CPUs will
195 * borrow CPU time from this CPU and cause RT task use > 95%
196 * CPU time. To make 'avoid starvation' work, takes a nap here.
197 */
198 if (unlikely(do_sleep))
199 schedule_timeout_killable(HZ * idle_pct / 100);
200
201 /* If an external event has set the need_resched flag, then
202 * we need to deal with it, or this loop will continue to
203 * spin without calling __mwait().
204 */
205 if (unlikely(need_resched()))
206 schedule();
207 }
208
209 exit_round_robin(tsk_index);
210 return 0;
211 }
212
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Peter Zijlstra
maintainerRe: [PATCH v5 08/13] ACPI: add perf low power callback
In reply to Stephane EranianQuoted 1 line · Stephane Eranian
This patch add an optional callback needed by some PMU features, e.g., AMD
"This patch" is a documented fail.
Quoted 29 lines
BRS, to give a chance to the
perf_eventscode to change its state before going to low power and after coming back.The callback is void when the
PERF_NEEDS_LOPWR_CBflag is not set. This flag must be set in arch specificperf_event.hheader whenever needed. When not set, there is no impact of the ACPI code.Signed-off-by Stephane Eranian <[email protected]>--- drivers/acpi/acpi_pad.c | 6 ++++++ drivers/acpi/processor_idle.c | 5 +++++ include/linux/perf_event.h | 6 ++++++ 3 files changed, 17 insertions(+)drivers/acpi/acpi_pad.c +3
-0@@ -164,6 +164,9 @@ static int power_saving_thread(void *data) tsc_marked_unstable =1 ; }local_irq_disable (); + +perf_lopwr_cb (true); +tick_broadcast_enable ();tick_broadcast_enter ();stop_critical_timings ();
drivers/acpi/acpi_pad.c static int power_saving_thread(void *data) · line 183
Quoted 9 lines
drivers/acpi/acpi_pad.c +3
-0@@ -172,6 +175,9 @@ static int power_saving_thread(void *data)start_critical_timings ();tick_broadcast_exit (); + +perf_lopwr_cb (false); +local_irq_enable ();if (time_before (expire_time, jiffies)) {
You forgot to Cc the maintainers of this stinking pile of poo. Can we please delete it instead?
Re: [PATCH v5 08/13] ACPI: add perf low power callback
Quoted 6 lines · Peter Zijlstra
Quoted 1 line · Stephane Eranian
This patch add an optional callback needed by some PMU features, e.g., AMD
"This patch" is a documented fail.
I understand this is not ideal. But the BRS feature must be disabled before going low-power.
Do you have a better proposal? Been trying to find the simplest way to support this with no
impact when not needed.
Quoted 41 lines
Quoted 29 lines
BRS, to give a chance to the
perf_eventscode to change its state before going to low power and after coming back.The callback is void when the
PERF_NEEDS_LOPWR_CBflag is not set. This flag must be set in arch specificperf_event.hheader whenever needed. When not set, there is no impact of the ACPI code.Signed-off-by Stephane Eranian <[email protected]>--- drivers/acpi/acpi_pad.c | 6 ++++++ drivers/acpi/processor_idle.c | 5 +++++ include/linux/perf_event.h | 6 ++++++ 3 files changed, 17 insertions(+)drivers/acpi/acpi_pad.c +3
-0@@ -164,6 +164,9 @@ static int power_saving_thread(void *data) tsc_marked_unstable =1 ; }local_irq_disable (); + +perf_lopwr_cb (true); +tick_broadcast_enable ();tick_broadcast_enter ();stop_critical_timings ();drivers/acpi/acpi_pad.c static int power_saving_thread(void *data) · line 183
Quoted 9 lines
drivers/acpi/acpi_pad.c +3
-0@@ -172,6 +175,9 @@ static int power_saving_thread(void *data)start_critical_timings ();tick_broadcast_exit (); + +perf_lopwr_cb (false); +local_irq_enable ();if (time_before (expire_time, jiffies)) {You forgot to Cc the maintainers of this stinking pile of poo. Can we please delete it instead?
Peter Zijlstra
maintainerRe: [PATCH v5 08/13] ACPI: add perf low power callback
Quoted 8 lines · Stephane Eranian
Quoted 6 lines · Peter Zijlstra
Quoted 1 line · Stephane Eranian
This patch add an optional callback needed by some PMU features, e.g., AMD
"This patch" is a documented fail.
I understand this is not ideal.
git grep -i "this patch" Documentation/process/submitting-patches.rst
"Describe your changes in imperative mood, e.g. "make xyzzy do frotz" instead of "[This patch] makes xyzzy do frotz" or "[I] changed xyzzy to do frotz", as if you are giving orders to the codebase to change its behaviour."
IOW, don't use "This patch" to start a Changelog (lot's of your patches here do so).
PATCH v5 9/13 perf/x86/amd: add idle hooks for branch sampling
In reply to Stephane EranianOn AMD Fam19h Zen3, the branch sampling (BRS) feature must be disabled before entering low power and re-enabled (if was active) when returning from low power. Otherwise, the NMI interrupt may be held up for too long and cause problems. Stopping BRS will cause the NMI to be delivered if it was held up.
The patch defines a perf_amd_brs_lopwr_cb() callback to stop/restart BRS. The callback is protected by a jump label which is enabled only when AMD BRS is detected. In all other cases, the callback is never called.
--- arch/x86/events/amd/brs.c | 32 +++++++++++++++++++++++++++++++ arch/x86/events/amd/core.c | 4 ++++ arch/x86/events/perf_event.h | 1 + arch/x86/include/asm/perf_event.h | 21 ++++++++++++++++++++ 4 files changed, 58 insertions(+)
arch/x86/events/amd/brs.c +32 -0
@@ -7,6 +7,7 @@* Contributed by Stephane Eranian <[email protected]> */ #include <linux/kernel.h> +#include <linux/jump_label.h> #include <asm/msr.h> #include <asm/cpufeature.h>
@@ -329,3 +330,34 @@ void amd_pmu_brs_sched_task(struct perf_event_context *ctx, bool sched_in)if (sched_in)amd_brs_poison_buffer (); } + +DEFINE_STATIC_KEY_FALSE (perf_lopwr_needed); + +/* +* called from ACPI processor_idle.c or acpi_pad.c +* with interrupts disabled +*/ +void perf_amd_brs_lopwr_cb (bool lopwr_in) +{ +struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +union amd_debug_extn_cfg cfg; + +/* +* on mwait in, we may end up in non C0 state. +* we must disable branch sampling to avoid holding the NMI +* for too long. We disable it in hardware but we +* keep the state in cpuc, so we can re-enable. +* +* The hardware will deliver the NMI if needed when brsmen cleared +*/ +if (cpuc->brs_active) { + cfg.val =get_debug_extn_cfg (); + cfg.brsmen = !lopwr_in; +set_debug_extn_cfg (cfg.val); + } +} + +void __init amd_brs_lopwr_init (void ) +{ +static_branch_enable (&perf_lopwr_needed); +}
arch/x86/events/amd/core.c +4 -0
@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0-only #include <linux/perf_event.h> +#include <linux/jump_label.h> #include <linux/export.h> #include <linux/types.h> #include <linux/init.h>
@@ -1189,6 +1190,9 @@ static int __init amd_core_pmu_init(void)* The put_event_constraints callback is shared with * Fam17h, set above */ + +/* branch sampling must be stopped when entering low power */ +amd_brs_lopwr_init (); } }
arch/x86/events/perf_event.h +1 -0
@@ -1226,6 +1226,7 @@ void amd_brs_enable(void);void amd_brs_enable_all (void );void amd_brs_disable_all (void );void amd_brs_drain (void ); +void amd_brs_lopwr_init (void );void amd_brs_disable_all (void );int amd_brs_setup_filter (struct perf_event *event);void amd_brs_reset (void );
arch/x86/include/asm/perf_event.h +21 -0
@@ -513,6 +513,27 @@ static inline void intel_pt_handle_vmx(int on)#if defined(CONFIG_PERF_EVENTS ) && defined(CONFIG_CPU_SUP_AMD )extern void amd_pmu_enable_virt (void );extern void amd_pmu_disable_virt (void ); + +#if defined(CONFIG_PERF_EVENTS_AMD_BRS ) + +#define PERF_NEEDS_LOPWR_CB 1 + +/* +* architectural low power callback impacts +* drivers/acpi/processor_idle.c +* drivers/acpi/acpi_pad.c +*/ +extern void perf_amd_brs_lopwr_cb (bool lopwr_in); +DECLARE_STATIC_KEY_FALSE (perf_lopwr_needed); + +static inline void perf_lopwr_cb (bool mode) +{ +/* key enabled only when BRS is available */ +if (static_branch_unlikely (&perf_lopwr_needed)) +perf_amd_brs_lopwr_cb (mode); +} +#endif /* PERF_NEEDS_LOPWR_CB */ +#else static inline void amd_pmu_enable_virt (void ) { }static inline void amd_pmu_disable_virt (void ) { }
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
Peter Zijlstra
maintainerRe: [PATCH v5 09/13] perf/x86/amd: add idle hooks for branch sampling
arch/x86/include/asm/perf_event.h static inline void intel_pt_handle_vmx(int on) · line 534
Quoted 6 lines · Stephane Eranian
arch/x86/include/asm/perf_event.h +6
-0+static inline void perf_lopwr_cb (bool mode) +{ +/* key enabled only when BRS is available */ +if (static_branch_unlikely (&perf_lopwr_needed)) +perf_amd_brs_lopwr_cb (mode); +}
static_call() might be better at this point..
Re: [PATCH v5 09/13] perf/x86/amd: add idle hooks for branch sampling
Quoted 10 lines · Peter Zijlstra
arch/x86/include/asm/perf_event.h static inline void intel_pt_handle_vmx(int on) · line 534
Quoted 6 lines · Stephane Eranian
arch/x86/include/asm/perf_event.h +6
-0+static inline void perf_lopwr_cb (bool mode) +{ +/* key enabled only when BRS is available */ +if (static_branch_unlikely (&perf_lopwr_needed)) +perf_amd_brs_lopwr_cb (mode); +}
static_call()might be better at this point..
Ok, I will fix this in v6.
Thanks.
PATCH v5 10/13 perf evsel: Make evsel__env always return a valid env
In reply to Stephane EranianFrom: Kim Phillips <[email protected]>
It's possible to have an evsel and evsel->evlist populated without an evsel->evlist->env, when, e.g., cmd_record is in its error path.
Future patches will add support for evsel__open_strerror to be able to customize error messaging based on perf_env__{arch,cpuid}, so let's have evsel__env return &perf_env instead of NULL in that case.
--- tools/perf/util/evsel.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/evsel.c +1 -1
@@ -2965,7 +2965,7 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,struct perf_env *evsel__env (struct evsel *evsel) { -if (evsel && evsel->evlist) +if (evsel && evsel->evlist && evsel->evlist->env)return evsel->evlist->env;return &perf_env; }
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
PATCH v5 11/13 perf tools: Improve IBS error handling
In reply to Stephane EranianFrom: Kim Phillips <[email protected]>
This patch improves the error message returned on failed perf_event_open() on AMD when using IBS.
Output of executing 'perf record -e ibs_op// true' BEFORE this patch:
The sys_perf_event_open() syscall returned with 22 (Invalid argument)for event (ibs_op//u). /bin/dmesg | grep -i perf may provide additional information.
Output after:
AMD IBS can't exclude kernel events. Try running at a higher privilege level.
Output of executing 'sudo perf record -e ibs_op// true' BEFORE this patch:
Error:
The sys_perf_event_open() syscall returned with 22 (Invalid argument) for event (ibs_op//). /bin/dmesg | grep -i perf may provide additional information.
Output after:
Error:
AMD IBS may only be available in system-wide/per-cpu mode. Try using -a, or -C and workload affinity
--- tools/perf/util/evsel.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
tools/perf/util/evsel.c +24 -0
@@ -2847,9 +2847,22 @@ static bool find_process(const char *name)return ret ? false : true; } +static bool is_amd (const char *arch,const char *cpuid) +{ +return arch && !strcmp ("x86" , arch) && cpuid &&strstarts (cpuid,"AuthenticAMD" ); +} + +static bool is_amd_ibs (struct evsel *evsel) +{ +return evsel->core.attr.precise_ip || !strncmp (evsel->pmu_name,"ibs" ,3 ); +} +int evsel__open_strerror (struct evsel *evsel,struct target *target,int err,char *msg,size_t size) { +struct perf_env *env =evsel__env (evsel); +const char *arch =perf_env__arch (env); +const char *cpuid =perf_env__cpuid (env);char sbuf[STRERR_BUFSIZE ];int printed =0 , enforced =0 ;
@@ -2949,6 +2962,17 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,return scnprintf (msg, size,"Invalid event (%s) in per-thread mode, enable system wide with '-a'." ,evsel__name (evsel)); +if (is_amd (arch, cpuid)) { +if (is_amd_ibs (evsel)) { +if (evsel->core.attr.exclude_kernel) +return scnprintf (msg, size, +"AMD IBS can't exclude kernel events. Try running at a higher privilege level." ); +if (!evsel->core.system_wide) +return scnprintf (msg, size, +"AMD IBS may only be available in system-wide/per-cpu mode. Try using -a, or -C and workload affinity" ); + } + } +break ;case ENODATA : return scnprintf(msg, size, "Cannot collect data source with the load latency event alone. "
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
PATCH v5 12/13 perf tools: Improve error handling of AMD Branch Sampling
In reply to Stephane EranianThis patch improves the error message printed by perf when perf_event_open() fails on AMD Zen3 when using the branch sampling feature. In the case of EINVAL, there are two main reasons: frequency mode or period is smaller than the depth of the branch sampling buffer (16). The patch checks the parameters of the call and tries to print a relevant message to explain the error:
$ perf record -b -e cpu/branch-brs/ -c 10 ls Error: AMD Branch Sampling does not support sampling period smaller than what is reported in /sys/devices/cpu/caps/branches.
$ perf record -b -e cpu/branch-brs/ ls Error: AMD Branch Sampling does not support frequency mode sampling, must pass a fixed sampling period via -c option or cpu/branch-brs,period=xxxx/.
--- tools/perf/util/evsel.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
tools/perf/util/evsel.c +14 -0
@@ -2857,6 +2857,12 @@ static bool is_amd_ibs(struct evsel *evsel)return evsel->core.attr.precise_ip || !strncmp (evsel->pmu_name,"ibs" ,3 ); } +static bool is_amd_brs (struct evsel *evsel) +{ +return ((evsel->core.attr.config &0xff ) ==0xc4 ) && + (evsel->core.attr.sample_type &PERF_SAMPLE_BRANCH_STACK ); +} +int evsel__open_strerror (struct evsel *evsel,struct target *target,int err,char *msg,size_t size) {
@@ -2971,6 +2977,14 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,return scnprintf (msg, size,"AMD IBS may only be available in system-wide/per-cpu mode. Try using -a, or -C and workload affinity" ); } +if (is_amd_brs (evsel)) { +if (evsel->core.attr.freq) +return scnprintf (msg, size, +"AMD Branch Sampling does not support frequency mode sampling, must pass a fixed sampling period via -c option or cpu/branch-brs,period=xxxx/." ); +/* another reason is that the period is too small */ +return scnprintf (msg, size, +"AMD Branch Sampling does not support sampling period smaller than what is reported in /sys/devices/cpu/caps/branches." ); + } }break ;
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
PATCH v5 13/13 perf report: add addr_from/addr_to sort dimensions
In reply to Stephane EranianThis patch adds a new pair of sorting dimensions used with branch sampling. With the existing symbol_from/symbol_to, branches captured in the same function would be collapsed into a single function if the latency (cycles) was all the same. That is the case on Intel Broadwell for instance. Since Intel Skylake, the latency is captured by hardware and therefore is used to disambiguate branches.
This patch adds addr_from/addr_to sort dimensions that will sort branches based on their addresses and not the function there are in. The output is still the function name but the offset within the function is provided to uniquely identify each branch. These new sort dimensions also help with annotate because they create different entries in the histogram which, in turn, generates proper branch annotation.
Here is an example using AMD's branch sampling:
$ perf record -a -b -c 1000037 -e cpu/branch-brs/ test_prg
$ perf report Samples: 6M of event 'cpu/branch-brs/', Event count (approx.): 6901276 Overhead Command Source Shared Object Source Symbol Target Symbol Basic Block Cycle 99.65% test_prg test_prg [.] test_thread [.] test_thread - 0.02% test_prg [kernel.vmlinux] [k] asm_sysvec_apic_timer_interrupt [k] error_entry -
$ perf report -F overhead,comm,dso,addr_from,addr_to Samples: 6M of event 'cpu/branch-brs/', Event count (approx.): 6901276 Overhead Command Shared Object Source Address Target Address 4.22% test_prg test_prg [.] test_thread+0x3c [.] test_thread+0x4 4.13% test_prg test_prg [.] test_thread+0x4 [.] test_thread+0x3a 4.09% test_prg test_prg [.] test_thread+0x3a [.] test_thread+0x6 4.08% test_prg test_prg [.] test_thread+0x2 [.] test_thread+0x3c 4.06% test_prg test_prg [.] test_thread+0x3e [.] test_thread+0x2 3.87% test_prg test_prg [.] test_thread+0x6 [.] test_thread+0x38 3.84% test_prg test_prg [.] test_thread [.] test_thread+0x3e 3.76% test_prg test_prg [.] test_thread+0x1e [.] test_thread 3.76% test_prg test_prg [.] test_thread+0x38 [.] test_thread+0x8 3.56% test_prg test_prg [.] test_thread+0x22 [.] test_thread+0x1e 3.54% test_prg test_prg [.] test_thread+0x8 [.] test_thread+0x36 3.47% test_prg test_prg [.] test_thread+0x1c [.] test_thread+0x22 3.45% test_prg test_prg [.] test_thread+0x36 [.] test_thread+0xa 3.28% test_prg test_prg [.] test_thread+0x24 [.] test_thread+0x1c 3.25% test_prg test_prg [.] test_thread+0xa [.] test_thread+0x34 3.24% test_prg test_prg [.] test_thread+0x1a [.] test_thread+0x24 3.20% test_prg test_prg [.] test_thread+0x34 [.] test_thread+0xc 3.04% test_prg test_prg [.] test_thread+0x26 [.] test_thread+0x1a 3.01% test_prg test_prg [.] test_thread+0xc [.] test_thread+0x32 2.98% test_prg test_prg [.] test_thread+0x18 [.] test_thread+0x26 2.94% test_prg test_prg [.] test_thread+0x32 [.] test_thread+0xe 2.76% test_prg test_prg [.] test_thread+0x28 [.] test_thread+0x18 2.73% test_prg test_prg [.] test_thread+0xe [.] test_thread+0x30 2.67% test_prg test_prg [.] test_thread+0x30 [.] test_thread+0x10 2.67% test_prg test_prg [.] test_thread+0x16 [.] test_thread+0x28 2.46% test_prg test_prg [.] test_thread+0x10 [.] test_thread+0x2e 2.44% test_prg test_prg [.] test_thread+0x2a [.] test_thread+0x16 2.38% test_prg test_prg [.] test_thread+0x14 [.] test_thread+0x2a 2.32% test_prg test_prg [.] test_thread+0x2e [.] test_thread+0x12 2.28% test_prg test_prg [.] test_thread+0x12 [.] test_thread+0x2c 2.16% test_prg test_prg [.] test_thread+0x2c [.] test_thread+0x14 0.02% test_prg [kernel.vmlinux] [k] asm_sysvec_apic_ti+0x5 [k] error_entry
--- tools/perf/util/hist.c | 2 + tools/perf/util/hist.h | 2 + tools/perf/util/sort.c | 128 +++++++++++++++++++++++++++++++++++++++++ tools/perf/util/sort.h | 2 + 4 files changed, 134 insertions(+)
tools/perf/util/hist.c +2 -0
@@ -124,6 +124,7 @@ void hists__calc_col_len(struct hists *hists, struct hist_entry *h) }else { symlen = unresolved_col_width +4 +2 ;hists__new_col_len (hists,HISTC_SYMBOL_FROM , symlen); +hists__new_col_len (hists,HISTC_ADDR_FROM , symlen);hists__set_unres_dso_col_len (hists,HISTC_DSO_FROM ); }
@@ -138,6 +139,7 @@ void hists__calc_col_len(struct hists *hists, struct hist_entry *h) }else { symlen = unresolved_col_width +4 +2 ;hists__new_col_len (hists,HISTC_SYMBOL_TO , symlen); +hists__new_col_len (hists,HISTC_ADDR_TO , symlen);hists__set_unres_dso_col_len (hists,HISTC_DSO_TO ); }
tools/perf/util/hist.h +2 -0
@@ -77,6 +77,8 @@ enum hist_column {HISTC_GLOBAL_INS_LAT ,HISTC_LOCAL_P_STAGE_CYC ,HISTC_GLOBAL_P_STAGE_CYC , +HISTC_ADDR_FROM , +HISTC_ADDR_TO , HISTC_NR_COLS, /* Last entry */ };
tools/perf/util/sort.c +128 -0
146 lines not on this page · raw message
tools/perf/util/sort.h +2 -0
@@ -251,6 +251,8 @@ enum sort_type {SORT_SRCLINE_FROM ,SORT_SRCLINE_TO ,SORT_SYM_IPC , +SORT_ADDR_FROM , +SORT_ADDR_TO ,/* memory mode specific sort keys */ __SORT_MEMORY_MODE,
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog