Skip to contentSkip to conversation

arm64/perf: Enable branch stack sampling

PATCH Superseded Anshuman Khandual 4 participants 18 messages Sep 29, 2022 – Oct 12, 2022

Subsystems (3)

v3 of 18 Superseded Latest is v18 →

lore
Share & tools
lore
#0 lore raw

v3 cover arm64/perf: Enable branch stack sampling

This series enables perf branch stack sampling support on arm64 platform via a new arch feature called Branch Record Buffer Extension (BRBE). All relevant register definitions could be accessed here.

https://developer.arm.com/documentation/ddi0601/2021-12/AArch64-Registers

This series applies on v6.0-rc5 after the BRBE related perf ABI changes series (V7) that was posted earlier, and a branch sample filter helper patch.

https://lore.kernel.org/all/[email protected]/

https://lore.kernel.org/all/[email protected]/

Changes in V3:

- Moved brbe_stack from the stack and now dynamically allocated
- Return PERF_BR_PRIV_UNKNOWN instead of -1 in brbe_fetch_perf_priv()
- Moved BRBIDR0, BRBCR, BRBFCR registers and fields into tools/sysreg
- Created dummy BRBINF_EL1 field definitions in tools/sysreg
- Dropped ARMPMU_EVT_PRIV framework which cached perfmon_capable()
- Both exception and exception return branche records are now captured only if the event has PERF_SAMPLE_BRANCH_KERNEL which would already been checked in generic perf via perf_allow_kernel()

Changes in V2:

https://lore.kernel.org/all/[email protected]/

- Dropped branch sample filter helpers consolidation patch from this series
- Added new hw_perf_event.flags element ARMPMU_EVT_PRIV to cache perfmon_capable()
- Use cached perfmon_capable() while configuring BRBE branch record filters

Changes in V1:

https://lore.kernel.org/linux-arm-kernel/[email protected]/

- Added CONFIG_PERF_EVENTS wrapper for all branch sample filter helpers
- Process new perf branch types via PERF_BR_EXTEND_ABI

Changes in RFC V2:

https://lore.kernel.org/linux-arm-kernel/[email protected]/

- Added branch_sample_priv() while consolidating other branch sample filter helpers
- Changed all SYS_BRBXXXN_EL1 register definition encodings per Marc
- Changed the BRBE driver as per proposed BRBE related perf ABI changes (V5)
- Added documentation for struct arm_pmu changes, updated commit message
- Updated commit message for BRBE detection infrastructure patch
- PERF_SAMPLE_BRANCH_KERNEL gets checked during arm event init (outside the driver)
- Branch privilege state capture mechanism has now moved inside the driver

Changes in RFC V1:

https://lore.kernel.org/all/[email protected]/

Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Mark Rutland <[email protected]>
Cc James Clark <[email protected]>
Cc Rob Herring <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Suzuki Poulose <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>

Anshuman Khandual (7):
arm64/perf: Add BRBE registers and fields
arm64/perf: Update struct arm_pmu for BRBE
arm64/perf: Update struct pmu_hw_events for BRBE driver/perf/arm_pmu_platform: Add support for BRBE attributes detection arm64/perf: Drive BRBE from perf event states
arm64/perf: Add BRBE driver
arm64/perf: Enable branch stack sampling

 arch/arm64/include/asm/sysreg.h | 107 ++++++++
 arch/arm64/kernel/perf_event.c  |  48 ++++
 arch/arm64/tools/sysreg         | 159 ++++++++++++
 drivers/perf/Kconfig            |  11 +
 drivers/perf/Makefile           |   1 +
 drivers/perf/arm_pmu.c          |  73 +++++-
 drivers/perf/arm_pmu_brbe.c     | 447 ++++++++++++++++++++++++++++++++
 drivers/perf/arm_pmu_brbe.h     | 259 ++++++++++++++++++
 drivers/perf/arm_pmu_platform.c |  34 +++
 include/linux/perf/arm_pmu.h    |  67 +++++
 10 files changed, 1203 insertions(+), 3 deletions(-)
 create mode 100644 drivers/perf/arm_pmu_brbe.c
 create mode 100644 drivers/perf/arm_pmu_brbe.h
Signature · 1 line
2.25.1
#1 lore raw

PATCH v3 1/7 arm64/perf: Add BRBE registers and fields

This adds BRBE related register definitions and various other related field macros there in. These will be used subsequently in a BRBE driver which is being added later on.

---
 arch/arm64/include/asm/sysreg.h | 107 +++++++++++++++++++++
 arch/arm64/tools/sysreg         | 159 ++++++++++++++++++++++++++++++++
 2 files changed, 266 insertions(+)
arch/arm64/include/asm/sysreg.h +107 -0
index 818df938a7ad..1cf9345730f0 100644
@@ -161,6 +161,109 @@
 #define SYS_DBGDTRTX_EL0		sys_reg(2, 3, 0, 5, 0)
 #define SYS_DBGVCR32_EL2		sys_reg(2, 4, 0, 7, 0)
 
+#define __SYS_BRBINFO(n)		sys_reg(2, 1, 8, ((n) & 0xf), ((((n) & 0x10)) >> 2 + 0))
+#define __SYS_BRBSRC(n)			sys_reg(2, 1, 8, ((n) & 0xf), ((((n) & 0x10)) >> 2 + 1))
+#define __SYS_BRBTGT(n)			sys_reg(2, 1, 8, ((n) & 0xf), ((((n) & 0x10)) >> 2 + 2))
+
+#define SYS_BRBINF0_EL1			__SYS_BRBINFO(0)
+#define SYS_BRBINF1_EL1			__SYS_BRBINFO(1)
+#define SYS_BRBINF2_EL1			__SYS_BRBINFO(2)
+#define SYS_BRBINF3_EL1			__SYS_BRBINFO(3)
+#define SYS_BRBINF4_EL1			__SYS_BRBINFO(4)
+#define SYS_BRBINF5_EL1			__SYS_BRBINFO(5)
+#define SYS_BRBINF6_EL1			__SYS_BRBINFO(6)
+#define SYS_BRBINF7_EL1			__SYS_BRBINFO(7)
+#define SYS_BRBINF8_EL1			__SYS_BRBINFO(8)
+#define SYS_BRBINF9_EL1			__SYS_BRBINFO(9)
+#define SYS_BRBINF10_EL1		__SYS_BRBINFO(10)
+#define SYS_BRBINF11_EL1		__SYS_BRBINFO(11)
+#define SYS_BRBINF12_EL1		__SYS_BRBINFO(12)
+#define SYS_BRBINF13_EL1		__SYS_BRBINFO(13)
+#define SYS_BRBINF14_EL1		__SYS_BRBINFO(14)
+#define SYS_BRBINF15_EL1		__SYS_BRBINFO(15)
+#define SYS_BRBINF16_EL1		__SYS_BRBINFO(16)
+#define SYS_BRBINF17_EL1		__SYS_BRBINFO(17)
+#define SYS_BRBINF18_EL1		__SYS_BRBINFO(18)
+#define SYS_BRBINF19_EL1		__SYS_BRBINFO(19)
+#define SYS_BRBINF20_EL1		__SYS_BRBINFO(20)
+#define SYS_BRBINF21_EL1		__SYS_BRBINFO(21)
+#define SYS_BRBINF22_EL1		__SYS_BRBINFO(22)
+#define SYS_BRBINF23_EL1		__SYS_BRBINFO(23)
+#define SYS_BRBINF24_EL1		__SYS_BRBINFO(24)
+#define SYS_BRBINF25_EL1		__SYS_BRBINFO(25)
+#define SYS_BRBINF26_EL1		__SYS_BRBINFO(26)
+#define SYS_BRBINF27_EL1		__SYS_BRBINFO(27)
+#define SYS_BRBINF28_EL1		__SYS_BRBINFO(28)
+#define SYS_BRBINF29_EL1		__SYS_BRBINFO(29)
+#define SYS_BRBINF30_EL1		__SYS_BRBINFO(30)
+#define SYS_BRBINF31_EL1		__SYS_BRBINFO(31)
+
+#define SYS_BRBSRC0_EL1			__SYS_BRBSRC(0)
+#define SYS_BRBSRC1_EL1			__SYS_BRBSRC(1)
+#define SYS_BRBSRC2_EL1			__SYS_BRBSRC(2)
+#define SYS_BRBSRC3_EL1			__SYS_BRBSRC(3)
+#define SYS_BRBSRC4_EL1			__SYS_BRBSRC(4)
+#define SYS_BRBSRC5_EL1			__SYS_BRBSRC(5)
+#define SYS_BRBSRC6_EL1			__SYS_BRBSRC(6)
+#define SYS_BRBSRC7_EL1			__SYS_BRBSRC(7)
+#define SYS_BRBSRC8_EL1			__SYS_BRBSRC(8)
+#define SYS_BRBSRC9_EL1			__SYS_BRBSRC(9)
+#define SYS_BRBSRC10_EL1		__SYS_BRBSRC(10)
+#define SYS_BRBSRC11_EL1		__SYS_BRBSRC(11)
+#define SYS_BRBSRC12_EL1		__SYS_BRBSRC(12)
+#define SYS_BRBSRC13_EL1		__SYS_BRBSRC(13)
+#define SYS_BRBSRC14_EL1		__SYS_BRBSRC(14)
+#define SYS_BRBSRC15_EL1		__SYS_BRBSRC(15)
+#define SYS_BRBSRC16_EL1		__SYS_BRBSRC(16)
+#define SYS_BRBSRC17_EL1		__SYS_BRBSRC(17)
+#define SYS_BRBSRC18_EL1		__SYS_BRBSRC(18)
+#define SYS_BRBSRC19_EL1		__SYS_BRBSRC(19)
+#define SYS_BRBSRC20_EL1		__SYS_BRBSRC(20)
+#define SYS_BRBSRC21_EL1		__SYS_BRBSRC(21)
+#define SYS_BRBSRC22_EL1		__SYS_BRBSRC(22)
+#define SYS_BRBSRC23_EL1		__SYS_BRBSRC(23)
+#define SYS_BRBSRC24_EL1		__SYS_BRBSRC(24)
+#define SYS_BRBSRC25_EL1		__SYS_BRBSRC(25)
+#define SYS_BRBSRC26_EL1		__SYS_BRBSRC(26)
+#define SYS_BRBSRC27_EL1		__SYS_BRBSRC(27)
+#define SYS_BRBSRC28_EL1		__SYS_BRBSRC(28)
+#define SYS_BRBSRC29_EL1		__SYS_BRBSRC(29)
+#define SYS_BRBSRC30_EL1		__SYS_BRBSRC(30)
+#define SYS_BRBSRC31_EL1		__SYS_BRBSRC(31)
+
+#define SYS_BRBTGT0_EL1			__SYS_BRBTGT(0)
+#define SYS_BRBTGT1_EL1			__SYS_BRBTGT(1)
+#define SYS_BRBTGT2_EL1			__SYS_BRBTGT(2)
+#define SYS_BRBTGT3_EL1			__SYS_BRBTGT(3)
+#define SYS_BRBTGT4_EL1			__SYS_BRBTGT(4)
+#define SYS_BRBTGT5_EL1			__SYS_BRBTGT(5)
+#define SYS_BRBTGT6_EL1			__SYS_BRBTGT(6)
+#define SYS_BRBTGT7_EL1			__SYS_BRBTGT(7)
+#define SYS_BRBTGT8_EL1			__SYS_BRBTGT(8)
+#define SYS_BRBTGT9_EL1			__SYS_BRBTGT(9)
+#define SYS_BRBTGT10_EL1		__SYS_BRBTGT(10)
+#define SYS_BRBTGT11_EL1		__SYS_BRBTGT(11)
+#define SYS_BRBTGT12_EL1		__SYS_BRBTGT(12)
+#define SYS_BRBTGT13_EL1		__SYS_BRBTGT(13)
+#define SYS_BRBTGT14_EL1		__SYS_BRBTGT(14)
+#define SYS_BRBTGT15_EL1		__SYS_BRBTGT(15)
+#define SYS_BRBTGT16_EL1		__SYS_BRBTGT(16)
+#define SYS_BRBTGT17_EL1		__SYS_BRBTGT(17)
+#define SYS_BRBTGT18_EL1		__SYS_BRBTGT(18)
+#define SYS_BRBTGT19_EL1		__SYS_BRBTGT(19)
+#define SYS_BRBTGT20_EL1		__SYS_BRBTGT(20)
+#define SYS_BRBTGT21_EL1		__SYS_BRBTGT(21)
+#define SYS_BRBTGT22_EL1		__SYS_BRBTGT(22)
+#define SYS_BRBTGT23_EL1		__SYS_BRBTGT(23)
+#define SYS_BRBTGT24_EL1		__SYS_BRBTGT(24)
+#define SYS_BRBTGT25_EL1		__SYS_BRBTGT(25)
+#define SYS_BRBTGT26_EL1		__SYS_BRBTGT(26)
+#define SYS_BRBTGT27_EL1		__SYS_BRBTGT(27)
+#define SYS_BRBTGT28_EL1		__SYS_BRBTGT(28)
+#define SYS_BRBTGT29_EL1		__SYS_BRBTGT(29)
+#define SYS_BRBTGT30_EL1		__SYS_BRBTGT(30)
+#define SYS_BRBTGT31_EL1		__SYS_BRBTGT(31)
+
 #define SYS_MIDR_EL1			sys_reg(3, 0, 0, 0, 0)
 #define SYS_MPIDR_EL1			sys_reg(3, 0, 0, 0, 5)
 #define SYS_REVIDR_EL1			sys_reg(3, 0, 0, 0, 6)
@@ -826,6 +929,7 @@
 #define ID_AA64MMFR2_CNP_SHIFT		0
 
 /* id_aa64dfr0 */
+#define ID_AA64DFR0_BRBE_SHIFT		52
 #define ID_AA64DFR0_MTPMU_SHIFT		48
 #define ID_AA64DFR0_TRBE_SHIFT		44
 #define ID_AA64DFR0_TRACE_FILT_SHIFT	40
@@ -848,6 +952,9 @@
 #define ID_AA64DFR0_PMSVER_8_2		0x1
 #define ID_AA64DFR0_PMSVER_8_3		0x2
 
+#define ID_AA64DFR0_BRBE		0x1
+#define ID_AA64DFR0_BRBE_V1P1		0x2
+
 #define ID_DFR0_PERFMON_SHIFT		24
 
 #define ID_DFR0_PERFMON_8_0		0x3
arch/arm64/tools/sysreg +159 -0

165 lines not on this page · raw message

Signature · 1 line
2.25.1
#2 lore raw

Re: [PATCH V3 1/7] arm64/perf: Add BRBE registers and fields

On Thu, Sep 29, 2022 at 01:28:51PM +0530, Anshuman Khandual wrote:

Thanks for doing this work - I did spot a few small issues though.

Quoted 6 lines

/* id_aa64dfr0 */

arch/arm64/include/asm/sysreg.h +1 -0
+#define ID_AA64DFR0_BRBE_SHIFT		52
 #define ID_AA64DFR0_MTPMU_SHIFT		48
 #define ID_AA64DFR0_TRBE_SHIFT		44
 #define ID_AA64DFR0_TRACE_FILT_SHIFT	40

arch/arm64/include/asm/sysreg.h line 960

Quoted 9 lines
arch/arm64/include/asm/sysreg.h +3 -0
@@ -848,6 +952,9 @@
 #define ID_AA64DFR0_PMSVER_8_2		0x1
 #define ID_AA64DFR0_PMSVER_8_3		0x2
 
+#define ID_AA64DFR0_BRBE		0x1
+#define ID_AA64DFR0_BRBE_V1P1		0x2
+
 #define ID_DFR0_PERFMON_SHIFT		24
 
 #define ID_DFR0_PERFMON_8_0		0x3

This is already done in -next as a result of ID_AA64DFR0_EL1 being converted, the enumberation define comes out as ID_AA64DFR0_EL1_BRBE_BRBE_V1P1.

arch/arm64/tools/sysreg line 58

Quoted 9 lines
arch/arm64/tools/sysreg +9 -0
+# This is just a dummy register declaration to get all common field masks and
+# shifts for accessing given BRBINF contents.
+Sysreg	BRBINF_EL1	2	1	8	0	0
+Res0	63:47
+Field	46	CCU
+Field	45:32	CC
+Res0	31:18
+Field	17	LASTFAILED
+Field	16	TX

According to DDI0487I.a bit 16 is called T not TX.

Review

Quoted 2 lines
(quoted diff) +2 -0
+Res0	15:14
+Enum	13:8		TYPE

It's probably worth noting in the comment the issue with Enums here that's meaning you're not using a SysregFields - I'm not sure what people will think of this but providing a definition using the ID for the 0th register does seem expedient.

arch/arm64/tools/sysreg line 85

Quoted 5 lines
arch/arm64/tools/sysreg +5 -0
+Enum	7:6	EL
+	0b00	EL0
+	0b01	EL1
+	0b10	EL2
+EndEnum

According to DDI0487I.a 0b11 has the value EL3 (when FEAT_BRBEv1p1).

arch/arm64/tools/sysreg line 102

Quoted 7 lines
arch/arm64/tools/sysreg +7 -0
+Sysreg	BRBCR_EL1	2	1	9	0	0
+Res0	63:24
+Field	23 	EXCEPTION
+Field	22 	ERTN
+Res0	21:9
+Field	8 	FZP
+Field	7	ZERO

According to DDI0487I.a bit 7 is Res0.

Quoted 1 line

+Field 2 ZERO1

According to DDI0487I.a bit 2 is Res0.

arch/arm64/tools/sysreg line 128

Quoted 3 lines
arch/arm64/tools/sysreg +2 -0
+Sysreg	BRBFCR_EL1	2	1	9	0	1
 
+Field	16	ENL

Accoding to DDI0487I.a this is EnI (ie, an L not an I).

arch/arm64/tools/sysreg line 145

Quoted 3 lines
arch/arm64/tools/sysreg +2 -0
+Sysreg	BRBINFINJ_EL1	2	1	9	1	0
 
+Field	16	TX

According to DDI0487I.a this is T not TX.

arch/arm64/tools/sysreg line 85

Quoted 5 lines
arch/arm64/tools/sysreg +5 -0
+Enum	7:6	EL
+	0b00	EL0
+	0b01	EL1
+	0b10	EL2
+EndEnum

According to DDI0487I.a 0b11 has the value EL3 (when FEAT_BRBEv1p1).

#3 lore raw

Re: [PATCH V3 1/7] arm64/perf: Add BRBE registers and fields

Quoted 18 lines · Mark Brown

On Thu, Sep 29, 2022 at 01:28:51PM +0530, Anshuman Khandual wrote:

Thanks for doing this work - I did spot a few small issues though.

Quoted 14 lines

/* id_aa64dfr0 */

arch/arm64/include/asm/sysreg.h +4 -0
+#define ID_AA64DFR0_BRBE_SHIFT		52
 #define ID_AA64DFR0_MTPMU_SHIFT		48
 #define ID_AA64DFR0_TRBE_SHIFT		44
 #define ID_AA64DFR0_TRACE_FILT_SHIFT	40
@@ -848,6 +952,9 @@
 #define ID_AA64DFR0_PMSVER_8_2		0x1
 #define ID_AA64DFR0_PMSVER_8_3		0x2
 
+#define ID_AA64DFR0_BRBE		0x1
+#define ID_AA64DFR0_BRBE_V1P1		0x2
+
 #define ID_DFR0_PERFMON_SHIFT		24
 
Quoted 5 lines

arch/arm64/include/asm/sysreg.h line 960

Quoted 1 line
arch/arm64/include/asm/sysreg.h +0 -0
 #define ID_DFR0_PERFMON_8_0		0x3

This is already done in -next as a result of ID_AA64DFR0_EL1 being converted, the enumberation define comes out as ID_AA64DFR0_EL1_BRBE_BRBE_V1P1.

Right. I will rebase the series on upcoming 6.1-rc1 which should have all the current -next patches including ID_AA64DFR0_EL1 migration into tools/sysreg. This should just fall in place.

Quoted 12 lines

arch/arm64/tools/sysreg line 58

Quoted 10 lines
arch/arm64/tools/sysreg +9 -0
+# This is just a dummy register declaration to get all common field masks and
+# shifts for accessing given BRBINF contents.
+Sysreg	BRBINF_EL1	2	1	8	0	0
+Res0	63:47
+Field	46	CCU
+Field	45:32	CC
+Res0	31:18
+Field	17	LASTFAILED
+Field	16	TX

According to DDI0487I.a bit 16 is called T not TX.

I understand :) The intention here was to keep the field name associated with "transaction" some how. But I guess, it would be more important to keep it as is matching the ARM ARM than something for readability purpose. Will change it as 'T'.

Quoted 6 lines

Review

Quoted 3 lines
(quoted diff) +2 -0
+Res0	15:14
+Enum	13:8		TYPE

It's probably worth noting in the comment the issue with Enums here that's meaning you're not using a SysregFields - I'm not sure what

Sure, will add a comment describing the problem of using enum elements inside SysregFields definition.

Quoted 2 lines

people will think of this but providing a definition using the ID for the 0th register does seem expedient.

I understand your concern but this turned out to be a better option

- Original sysreg.h based definitions had all field masks on the right end

- When reading (reg >> field_shift) & field_mask
- When writing (val & field_mask) << field_shift

- tools/sysreg format creates in-place field masks

- When reading (reg & field_mask) >> field_shift
- When writing (val << field_shift) & field_mask

- After moving some BRBE registers into tools/sysreg, the driver code had to be changed to accommodate these new write/read methods

- To avoid mix up in the BRBE driver, all BRBINF register fields need to be converted into in place masks, either in sysreg.h itself or moving into tools/sysreg

Moving BRBE fields into tools/sysreg via a dummy BRBINF_EL1 register seems to achieve that objective. These common fields can be used to work on any BRBINF<N>_EL1 register value. But I might just keep them in sysreg.h, if the proposed solution is not preferable or seems expedient.

Later when enum support comes up in SysregFields and tools/sysreg supports formula based crm/op2 expansion entire BRBINF, BRBSRC, BRBTGT register set can be moved into tools/sysreg.

Quoted 8 lines

arch/arm64/tools/sysreg line 85

Quoted 6 lines
arch/arm64/tools/sysreg +5 -0
+Enum	7:6	EL
+	0b00	EL0
+	0b01	EL1
+	0b10	EL2
+EndEnum

According to DDI0487I.a 0b11 has the value EL3 (when FEAT_BRBEv1p1).

Sure, will add it.

Quoted 10 lines

arch/arm64/tools/sysreg line 102

Quoted 8 lines
arch/arm64/tools/sysreg +7 -0
+Sysreg	BRBCR_EL1	2	1	9	0	0
+Res0	63:24
+Field	23 	EXCEPTION
+Field	22 	ERTN
+Res0	21:9
+Field	8 	FZP
+Field	7	ZERO

According to DDI0487I.a bit 7 is Res0.

Sure, will change.

Quoted 4 lines
Quoted 2 lines

+Field 2 ZERO1

According to DDI0487I.a bit 2 is Res0.

Sure, will change.

Quoted 6 lines

arch/arm64/tools/sysreg line 128

Quoted 4 lines
arch/arm64/tools/sysreg +2 -0
+Sysreg	BRBFCR_EL1	2	1	9	0	1
 
+Field	16	ENL

Accoding to DDI0487I.a this is EnI (ie, an L not an I).

ENL != Enl ? Do we need to match the case as well ?

Quoted 6 lines

arch/arm64/tools/sysreg line 145

Quoted 4 lines
arch/arm64/tools/sysreg +2 -0
+Sysreg	BRBINFINJ_EL1	2	1	9	1	0
 
+Field	16	TX

According to DDI0487I.a this is T not TX.

As mentioned, will change it as 'T'.

Quoted 8 lines

arch/arm64/tools/sysreg line 85

Quoted 6 lines
arch/arm64/tools/sysreg +5 -0
+Enum	7:6	EL
+	0b00	EL0
+	0b01	EL1
+	0b10	EL2
+EndEnum

According to DDI0487I.a 0b11 has the value EL3 (when FEAT_BRBEv1p1).

Sure, will add.

#4 lore raw

PATCH v3 2/7 arm64/perf: Update struct arm_pmu for BRBE

In reply to Anshuman Khandual

Although BRBE is an armv8 speciifc HW feature, abstracting out its various function callbacks at the struct arm_pmu level is preferred, as it cleaner , easier to follow and maintain.

Besides some helpers i.e brbe_supported(), brbe_probe() and brbe_reset() might not fit seamlessly, when tried to be embedded via existing arm_pmu helpers in the armv8 implementation.

Updates the struct arm_pmu to include all required helpers that will drive BRBE functionality for a given PMU implementation. These are the following.

- brbe_filter : Convert perf event filters into BRBE HW filters
- brbe_probe : Probe BRBE HW and capture its attributes
- brbe_enable : Enable BRBE HW with a given config
- brbe_disable : Disable BRBE HW
- brbe_read : Read BRBE buffer for captured branch records
- brbe_reset : Reset BRBE buffer
- brbe_supported: Whether BRBE is supported or not

A BRBE driver implementation needs to provide these functionalities.

---
 arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++++++++++++++++++
 include/linux/perf/arm_pmu.h   | 21 ++++++++++++++++++++
 2 files changed, 57 insertions(+)
arch/arm64/kernel/perf_event.c +36 -0
index cb69ff1e6138..e7013699171f 100644
@@ -1025,6 +1025,35 @@ static int armv8pmu_filter_match(struct perf_event *event)
 	return evtype != ARMV8_PMUV3_PERFCTR_CHAIN;
 }
 
+static void armv8pmu_brbe_filter(struct pmu_hw_events *hw_event, struct perf_event *event)
+{
+}
+
+static void armv8pmu_brbe_enable(struct pmu_hw_events *hw_event)
+{
+}
+
+static void armv8pmu_brbe_disable(struct pmu_hw_events *hw_event)
+{
+}
+
+static void armv8pmu_brbe_read(struct pmu_hw_events *hw_event, struct perf_event *event)
+{
+}
+
+static void armv8pmu_brbe_probe(struct pmu_hw_events *hw_event)
+{
+}
+
+static void armv8pmu_brbe_reset(struct pmu_hw_events *hw_event)
+{
+}
+
+static bool armv8pmu_brbe_supported(struct perf_event *event)
+{
+	return false;
+}
+
 static void armv8pmu_reset(void *info)
 {
 	struct arm_pmu *cpu_pmu = (struct arm_pmu *)info;
@@ -1257,6 +1286,13 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
 
 	cpu_pmu->pmu.event_idx		= armv8pmu_user_event_idx;
 
+	cpu_pmu->brbe_filter		= armv8pmu_brbe_filter;
+	cpu_pmu->brbe_enable		= armv8pmu_brbe_enable;
+	cpu_pmu->brbe_disable		= armv8pmu_brbe_disable;
+	cpu_pmu->brbe_read		= armv8pmu_brbe_read;
+	cpu_pmu->brbe_probe		= armv8pmu_brbe_probe;
+	cpu_pmu->brbe_reset		= armv8pmu_brbe_reset;
+	cpu_pmu->brbe_supported		= armv8pmu_brbe_supported;
 	cpu_pmu->name			= name;
 	cpu_pmu->map_event		= map_event;
 	cpu_pmu->attr_groups[ARMPMU_ATTR_GROUP_EVENTS] = events ?
include/linux/perf/arm_pmu.h +21 -0
index 0407a38b470a..3d427ac0ca45 100644
@@ -100,6 +100,27 @@ struct arm_pmu {
 	void		(*reset)(void *);
 	int		(*map_event)(struct perf_event *event);
 	int		(*filter_match)(struct perf_event *event);
+
+	/* Convert perf event filters into BRBE HW filters */
+	void		(*brbe_filter)(struct pmu_hw_events *hw_events, struct perf_event *event);
+
+	/* Probe BRBE HW and capture its attributes */
+	void		(*brbe_probe)(struct pmu_hw_events *hw_events);
+
+	/* Enable BRBE HW with a given config */
+	void		(*brbe_enable)(struct pmu_hw_events *hw_events);
+
+	/* Disable BRBE HW */
+	void		(*brbe_disable)(struct pmu_hw_events *hw_events);
+
+	/* Process BRBE buffer for captured branch records */
+	void		(*brbe_read)(struct pmu_hw_events *hw_events, struct perf_event *event);
+
+	/* Reset BRBE buffer */
+	void		(*brbe_reset)(struct pmu_hw_events *hw_events);
+
+	/* Check whether BRBE is supported */
+	bool		(*brbe_supported)(struct perf_event *event);
 	int		num_events;
 	bool		secure_access; /* 32-bit ARM only */
 #define ARMV8_PMUV3_MAX_COMMON_EVENTS		0x40
Signature · 1 line
2.25.1
#5 lore raw

PATCH v3 3/7 arm64/perf: Update struct pmu_hw_events for BRBE

In reply to Anshuman Khandual

A single perf event instance BRBE related contexts and data will be tracked in struct pmu_hw_events. Hence update the structure to accommodate required details related to BRBE.

---
 drivers/perf/arm_pmu.c       |  1 +
 include/linux/perf/arm_pmu.h | 26 ++++++++++++++++++++++++++
 2 files changed, 27 insertions(+)
drivers/perf/arm_pmu.c +1 -0
index 59d3980b8ca2..16fda9a1229f 100644
@@ -905,6 +905,7 @@ static struct arm_pmu *__armpmu_alloc(gfp_t flags)
 
 		events = per_cpu_ptr(pmu->hw_events, cpu);
 		raw_spin_lock_init(&events->pmu_lock);
+		events->branches = kmalloc(sizeof(struct brbe_records), flags);
 		events->percpu_pmu = pmu;
 	}
 
include/linux/perf/arm_pmu.h +26 -0
index 3d427ac0ca45..ffce43ceb670 100644
@@ -43,6 +43,16 @@
 	},								\
 }
 
+/*
+ * Maximum branch records in BRBE
+ */
+#define BRBE_MAX_ENTRIES 64
+
+struct brbe_records {
+	struct perf_branch_stack	brbe_stack;
+	struct perf_branch_entry	brbe_entries[BRBE_MAX_ENTRIES];
+};
+
 /* The events for a given PMU register set. */
 struct pmu_hw_events {
 	/*
@@ -69,6 +79,22 @@ struct pmu_hw_events {
 	struct arm_pmu		*percpu_pmu;
 
 	int irq;
+
+	/* Detected BRBE attributes */
+	bool				v1p1;
+	int				brbe_cc;
+	int				brbe_nr;
+
+	/* Evaluated BRBE configuration */
+	u64				brbfcr;
+	u64				brbcr;
+
+	/* Tracked BRBE context */
+	unsigned int			brbe_users;
+	void				*brbe_context;
+
+	/* Captured BRBE buffer - copied as is into perf_sample_data */
+	struct brbe_records		*branches;
 };
 
 enum armpmu_attr_groups {
Signature · 1 line
2.25.1
#6 lore raw

PATCH v3 4/7 driver/perf/arm_pmu_platform: Add support for BRBE attributes detection

In reply to Anshuman Khandual

This adds arm pmu infrastrure to probe BRBE implementation's attributes via driver exported callbacks later. The actual BRBE feature detection will be added by the driver itself.

CPU specific BRBE entries, cycle count, format support gets detected during PMU init. This information gets saved in per-cpu struct pmu_hw_events which later helps in operating BRBE during a perf event context.

---
 drivers/perf/arm_pmu_platform.c | 34 +++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
drivers/perf/arm_pmu_platform.c +34 -0
index 933b96e243b8..acdc445081aa 100644
@@ -172,6 +172,36 @@ static int armpmu_request_irqs(struct arm_pmu *armpmu)
 	return err;
 }
 
+static void arm_brbe_probe_cpu(void *info)
+{
+	struct pmu_hw_events *hw_events;
+	struct arm_pmu *armpmu = info;
+
+	/*
+	 * Return from here, if BRBE driver has not been
+	 * implemented for this PMU. This helps prevent
+	 * kernel crash later when brbe_probe() will be
+	 * called on the PMU.
+	 */
+	if (!armpmu->brbe_probe)
+		return;
+
+	hw_events = per_cpu_ptr(armpmu->hw_events, smp_processor_id());
+	armpmu->brbe_probe(hw_events);
+}
+
+static int armpmu_request_brbe(struct arm_pmu *armpmu)
+{
+	int cpu, err = 0;
+
+	for_each_cpu(cpu, &armpmu->supported_cpus) {
+		err = smp_call_function_single(cpu, arm_brbe_probe_cpu, armpmu, 1);
+		if (err)
+			return err;
+	}
+	return err;
+}
+
 static void armpmu_free_irqs(struct arm_pmu *armpmu)
 {
 	int cpu;
@@ -229,6 +259,10 @@ int arm_pmu_device_probe(struct platform_device *pdev,
 	if (ret)
 		goto out_free_irqs;
 
+	ret = armpmu_request_brbe(pmu);
+	if (ret)
+		goto out_free_irqs;
+
 	ret = armpmu_register(pmu);
 	if (ret) {
 		dev_err(dev, "failed to register PMU devices!\n");
Signature · 1 line
2.25.1
#7 lore raw

Re: [PATCH V3 4/7] driver/perf/arm_pmu_platform: Add support for BRBE attributes detection

Quoted 39 lines · Anshuman Khandual

This adds arm pmu infrastrure to probe BRBE implementation's attributes via driver exported callbacks later. The actual BRBE feature detection will be added by the driver itself.

CPU specific BRBE entries, cycle count, format support gets detected during PMU init. This information gets saved in per-cpu struct pmu_hw_events which later helps in operating BRBE during a perf event context.

---
 drivers/perf/arm_pmu_platform.c | 34 +++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
drivers/perf/arm_pmu_platform.c +14 -0
index 933b96e243b8..acdc445081aa 100644
@@ -172,6 +172,36 @@ static int armpmu_request_irqs(struct arm_pmu *armpmu)
 	return err;
 }
 
+static void arm_brbe_probe_cpu(void *info)
+{
+	struct pmu_hw_events *hw_events;
+	struct arm_pmu *armpmu = info;
+
+	/*
+	 * Return from here, if BRBE driver has not been
+	 * implemented for this PMU. This helps prevent
+	 * kernel crash later when brbe_probe() will be
+	 * called on the PMU.
+	 */
+	if (!armpmu->brbe_probe)
+		return;
+

drivers/perf/arm_pmu_platform.c static int armpmu_request_irqs(struct arm_pmu *armpmu) · line 198

Quoted 10 lines
drivers/perf/arm_pmu_platform.c +10 -0
+	hw_events = per_cpu_ptr(armpmu->hw_events, smp_processor_id());
+	armpmu->brbe_probe(hw_events);
+}
+
+static int armpmu_request_brbe(struct arm_pmu *armpmu)
+{
+	int cpu, err = 0;
+
+	for_each_cpu(cpu, &armpmu->supported_cpus) {
+		err = smp_call_function_single(cpu, arm_brbe_probe_cpu, armpmu, 1);

Hi Anshuman,

I have LOCKDEP on and the patchset applied to perf/core (82aad7ff7) on
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git and I get this:

armv8-pmu pmu: hw perfevents: no interrupt-affinity property, guessing.
brbe: implementation found on cpu 0

=============================
[ BUG: Invalid wait context ]
6.0.0-rc7 #38 Not tainted
-----------------------------
kworker/u8:0/9 is trying to lock:
ffff000800855898 (&port_lock_key){....}-{3:3}, at:

pl011_console_write+0x148/0x240
   other info that might help us debug this:
   context-{2:2}
   5 locks held by kworker/u8:0/9:
    #0: ffff00080032a138 ((wq_completion)eval_map_wq){+.+.}-{0:0}, at:
process_one_work+0x200/0x6b0
    #1: ffff80000807bde0
((work_completion)(&eval_map_work)){+.+.}-{0:0}, at:
process_one_work+0x200/0x6b0
    #2: ffff80000aa3db70 (trace_event_sem){+.+.}-{4:4}, at:
trace_event_eval_update+0x28/0x420
    #3: ffff80000a9afe58 (console_lock){+.+.}-{0:0}, at:
vprintk_emit+0x130/0x380
    #4: ffff80000a9aff78 (console_owner){-...}-{0:0}, at:
console_emit_next_record.constprop.0+0x128/0x338
   stack backtrace:
   CPU: 0 PID: 9 Comm: kworker/u8:0 Not tainted 6.0.0-rc7 #38
   Hardware name: Foundation-v8A (DT)
   Workqueue: eval_map_wq eval_map_work_func
   Call trace:
    dump_backtrace+0x114/0x120
    show_stack+0x20/0x58
    dump_stack_lvl+0x9c/0xd8
    dump_stack+0x18/0x34
    __lock_acquire+0x17cc/0x1920
    lock_acquire+0x138/0x3b8
    _raw_spin_lock+0x58/0x70
    pl011_console_write+0x148/0x240
    console_emit_next_record.constprop.0+0x194/0x338
    console_unlock+0x18c/0x208
    vprintk_emit+0x24c/0x380
    vprintk_default+0x40/0x50
    vprintk+0xd4/0xf0
    _printk+0x68/0x90
    arm64_pmu_brbe_probe+0x10c/0x128
    armv8pmu_brbe_probe+0x18/0x28
    arm_brbe_probe_cpu+0x44/0x58
    __flush_smp_call_function_queue+0x1d0/0x440
    generic_smp_call_function_single_interrupt+0x20/0x78
    ipi_handler+0x98/0x368
    handle_percpu_devid_irq+0xc0/0x3a8
    generic_handle_domain_irq+0x34/0x50
    gic_handle_irq+0x58/0x138
    call_on_irq_stack+0x2c/0x58
    do_interrupt_handler+0x88/0x90
    el1_interrupt+0x40/0x78
    el1h_64_irq_handler+0x18/0x28
    el1h_64_irq+0x64/0x68
    trace_event_eval_update+0x114/0x420
    eval_map_work_func+0x30/0x40
    process_one_work+0x298/0x6b0
    worker_thread+0x54/0x408
    kthread+0x118/0x128
    ret_from_fork+0x10/0x20
   brbe: implementation found on cpu 1
   brbe: implementation found on cpu 2
   brbe: implementation found on cpu 3
Quoted 20 lines
drivers/perf/arm_pmu_platform.c +10 -0
+		if (err)
+			return err;
+	}
+	return err;
+}
+
 static void armpmu_free_irqs(struct arm_pmu *armpmu)
 {
 	int cpu;
@@ -229,6 +259,10 @@ int arm_pmu_device_probe(struct platform_device *pdev,
 	if (ret)
 		goto out_free_irqs;
 
+	ret = armpmu_request_brbe(pmu);
+	if (ret)
+		goto out_free_irqs;
+
 	ret = armpmu_register(pmu);
 	if (ret) {
 		dev_err(dev, "failed to register PMU devices!\n");
#8 lore raw

Re: [PATCH V3 4/7] driver/perf/arm_pmu_platform: Add support for BRBE attributes detection

Quoted 55 lines · James Clark
Quoted 39 lines · Anshuman Khandual

This adds arm pmu infrastrure to probe BRBE implementation's attributes via driver exported callbacks later. The actual BRBE feature detection will be added by the driver itself.

CPU specific BRBE entries, cycle count, format support gets detected during PMU init. This information gets saved in per-cpu struct pmu_hw_events which later helps in operating BRBE during a perf event context.

---
 drivers/perf/arm_pmu_platform.c | 34 +++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
drivers/perf/arm_pmu_platform.c +14 -0
index 933b96e243b8..acdc445081aa 100644
@@ -172,6 +172,36 @@ static int armpmu_request_irqs(struct arm_pmu *armpmu)
 	return err;
 }
 
+static void arm_brbe_probe_cpu(void *info)
+{
+	struct pmu_hw_events *hw_events;
+	struct arm_pmu *armpmu = info;
+
+	/*
+	 * Return from here, if BRBE driver has not been
+	 * implemented for this PMU. This helps prevent
+	 * kernel crash later when brbe_probe() will be
+	 * called on the PMU.
+	 */
+	if (!armpmu->brbe_probe)
+		return;
+

drivers/perf/arm_pmu_platform.c static int armpmu_request_irqs(struct arm_pmu *armpmu) · line 198

Quoted 10 lines
drivers/perf/arm_pmu_platform.c +10 -0
+	hw_events = per_cpu_ptr(armpmu->hw_events, smp_processor_id());
+	armpmu->brbe_probe(hw_events);
+}
+
+static int armpmu_request_brbe(struct arm_pmu *armpmu)
+{
+	int cpu, err = 0;
+
+	for_each_cpu(cpu, &armpmu->supported_cpus) {
+		err = smp_call_function_single(cpu, arm_brbe_probe_cpu, armpmu, 1);

Hi Anshuman,

Quoted 2 lines

I have LOCKDEP on and the patchset applied to perf/core (82aad7ff7) on
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git and I get

Can you confirm if this is currently the correct place to apply this to? I'm only getting 0 length branch stacks now. Seems like it could be something to do with the layout of perf samples because I know that was done in separate commits:

sudo ./perf record -j any_call -- ls
./perf report -D | grep "branch stack"
... branch stack: nr:0
... branch stack: nr:0
... branch stack: nr:0
... branch stack: nr:0
...

Quoted 89 lines

this:

armv8-pmu pmu: hw perfevents: no interrupt-affinity property, guessing.
brbe: implementation found on cpu 0

   =============================
   [ BUG: Invalid wait context ]
   6.0.0-rc7 #38 Not tainted
   -----------------------------
   kworker/u8:0/9 is trying to lock:
   ffff000800855898 (&port_lock_key){....}-{3:3}, at:
pl011_console_write+0x148/0x240
   other info that might help us debug this:
   context-{2:2}
   5 locks held by kworker/u8:0/9:
    #0: ffff00080032a138 ((wq_completion)eval_map_wq){+.+.}-{0:0}, at:
process_one_work+0x200/0x6b0
    #1: ffff80000807bde0
((work_completion)(&eval_map_work)){+.+.}-{0:0}, at:
process_one_work+0x200/0x6b0
    #2: ffff80000aa3db70 (trace_event_sem){+.+.}-{4:4}, at:
trace_event_eval_update+0x28/0x420
    #3: ffff80000a9afe58 (console_lock){+.+.}-{0:0}, at:
vprintk_emit+0x130/0x380
    #4: ffff80000a9aff78 (console_owner){-...}-{0:0}, at:
console_emit_next_record.constprop.0+0x128/0x338
   stack backtrace:
   CPU: 0 PID: 9 Comm: kworker/u8:0 Not tainted 6.0.0-rc7 #38
   Hardware name: Foundation-v8A (DT)
   Workqueue: eval_map_wq eval_map_work_func
   Call trace:
    dump_backtrace+0x114/0x120
    show_stack+0x20/0x58
    dump_stack_lvl+0x9c/0xd8
    dump_stack+0x18/0x34
    __lock_acquire+0x17cc/0x1920
    lock_acquire+0x138/0x3b8
    _raw_spin_lock+0x58/0x70
    pl011_console_write+0x148/0x240
    console_emit_next_record.constprop.0+0x194/0x338
    console_unlock+0x18c/0x208
    vprintk_emit+0x24c/0x380
    vprintk_default+0x40/0x50
    vprintk+0xd4/0xf0
    _printk+0x68/0x90
    arm64_pmu_brbe_probe+0x10c/0x128
    armv8pmu_brbe_probe+0x18/0x28
    arm_brbe_probe_cpu+0x44/0x58
    __flush_smp_call_function_queue+0x1d0/0x440
    generic_smp_call_function_single_interrupt+0x20/0x78
    ipi_handler+0x98/0x368
    handle_percpu_devid_irq+0xc0/0x3a8
    generic_handle_domain_irq+0x34/0x50
    gic_handle_irq+0x58/0x138
    call_on_irq_stack+0x2c/0x58
    do_interrupt_handler+0x88/0x90
    el1_interrupt+0x40/0x78
    el1h_64_irq_handler+0x18/0x28
    el1h_64_irq+0x64/0x68
    trace_event_eval_update+0x114/0x420
    eval_map_work_func+0x30/0x40
    process_one_work+0x298/0x6b0
    worker_thread+0x54/0x408
    kthread+0x118/0x128
    ret_from_fork+0x10/0x20
   brbe: implementation found on cpu 1
   brbe: implementation found on cpu 2
   brbe: implementation found on cpu 3
Quoted 20 lines
drivers/perf/arm_pmu_platform.c +10 -0
+		if (err)
+			return err;
+	}
+	return err;
+}
+
 static void armpmu_free_irqs(struct arm_pmu *armpmu)
 {
 	int cpu;
@@ -229,6 +259,10 @@ int arm_pmu_device_probe(struct platform_device *pdev,
 	if (ret)
 		goto out_free_irqs;
 
+	ret = armpmu_request_brbe(pmu);
+	if (ret)
+		goto out_free_irqs;
+
 	ret = armpmu_register(pmu);
 	if (ret) {
 		dev_err(dev, "failed to register PMU devices!\n");
#9 lore raw

Re: [PATCH V3 4/7] driver/perf/arm_pmu_platform: Add support for BRBE attributes detection

Quoted 58 lines · James Clark
Quoted 55 lines · James Clark
Quoted 39 lines · Anshuman Khandual

This adds arm pmu infrastrure to probe BRBE implementation's attributes via driver exported callbacks later. The actual BRBE feature detection will be added by the driver itself.

CPU specific BRBE entries, cycle count, format support gets detected during PMU init. This information gets saved in per-cpu struct pmu_hw_events which later helps in operating BRBE during a perf event context.

---
 drivers/perf/arm_pmu_platform.c | 34 +++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
drivers/perf/arm_pmu_platform.c +14 -0
index 933b96e243b8..acdc445081aa 100644
@@ -172,6 +172,36 @@ static int armpmu_request_irqs(struct arm_pmu *armpmu)
 	return err;
 }
 
+static void arm_brbe_probe_cpu(void *info)
+{
+	struct pmu_hw_events *hw_events;
+	struct arm_pmu *armpmu = info;
+
+	/*
+	 * Return from here, if BRBE driver has not been
+	 * implemented for this PMU. This helps prevent
+	 * kernel crash later when brbe_probe() will be
+	 * called on the PMU.
+	 */
+	if (!armpmu->brbe_probe)
+		return;
+

drivers/perf/arm_pmu_platform.c static int armpmu_request_irqs(struct arm_pmu *armpmu) · line 198

Quoted 10 lines
drivers/perf/arm_pmu_platform.c +10 -0
+	hw_events = per_cpu_ptr(armpmu->hw_events, smp_processor_id());
+	armpmu->brbe_probe(hw_events);
+}
+
+static int armpmu_request_brbe(struct arm_pmu *armpmu)
+{
+	int cpu, err = 0;
+
+	for_each_cpu(cpu, &armpmu->supported_cpus) {
+		err = smp_call_function_single(cpu, arm_brbe_probe_cpu, armpmu, 1);

Hi Anshuman,

Quoted 4 lines
Quoted 2 lines

I have LOCKDEP on and the patchset applied to perf/core (82aad7ff7) on
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git and I get

Can you confirm if this is currently the correct place to apply this to?

This series applied on v6.0-rc5 after the perf ABI changes, both in kernel and in user space tools.

Quoted 3 lines

I'm only getting 0 length branch stacks now. Seems like it could be something to do with the layout of perf samples because I know that was done in separate commits:

Right, might be.

Quoted 7 lines

sudo ./perf record -j any_call -- ls
./perf report -D | grep "branch stack"
... branch stack: nr:0
... branch stack: nr:0
... branch stack: nr:0
... branch stack: nr:0

I am planning to respin the series on 6.1-rc1 next week which should solve these multiple moving parts problem.

Quoted 91 lines

...

Quoted 89 lines

this:

armv8-pmu pmu: hw perfevents: no interrupt-affinity property, guessing.
brbe: implementation found on cpu 0

   =============================
   [ BUG: Invalid wait context ]
   6.0.0-rc7 #38 Not tainted
   -----------------------------
   kworker/u8:0/9 is trying to lock:
   ffff000800855898 (&port_lock_key){....}-{3:3}, at:
pl011_console_write+0x148/0x240
   other info that might help us debug this:
   context-{2:2}
   5 locks held by kworker/u8:0/9:
    #0: ffff00080032a138 ((wq_completion)eval_map_wq){+.+.}-{0:0}, at:
process_one_work+0x200/0x6b0
    #1: ffff80000807bde0
((work_completion)(&eval_map_work)){+.+.}-{0:0}, at:
process_one_work+0x200/0x6b0
    #2: ffff80000aa3db70 (trace_event_sem){+.+.}-{4:4}, at:
trace_event_eval_update+0x28/0x420
    #3: ffff80000a9afe58 (console_lock){+.+.}-{0:0}, at:
vprintk_emit+0x130/0x380
    #4: ffff80000a9aff78 (console_owner){-...}-{0:0}, at:
console_emit_next_record.constprop.0+0x128/0x338
   stack backtrace:
   CPU: 0 PID: 9 Comm: kworker/u8:0 Not tainted 6.0.0-rc7 #38
   Hardware name: Foundation-v8A (DT)
   Workqueue: eval_map_wq eval_map_work_func
   Call trace:
    dump_backtrace+0x114/0x120
    show_stack+0x20/0x58
    dump_stack_lvl+0x9c/0xd8
    dump_stack+0x18/0x34
    __lock_acquire+0x17cc/0x1920
    lock_acquire+0x138/0x3b8
    _raw_spin_lock+0x58/0x70
    pl011_console_write+0x148/0x240
    console_emit_next_record.constprop.0+0x194/0x338
    console_unlock+0x18c/0x208
    vprintk_emit+0x24c/0x380
    vprintk_default+0x40/0x50
    vprintk+0xd4/0xf0
    _printk+0x68/0x90
    arm64_pmu_brbe_probe+0x10c/0x128
    armv8pmu_brbe_probe+0x18/0x28
    arm_brbe_probe_cpu+0x44/0x58
    __flush_smp_call_function_queue+0x1d0/0x440
    generic_smp_call_function_single_interrupt+0x20/0x78
    ipi_handler+0x98/0x368
    handle_percpu_devid_irq+0xc0/0x3a8
    generic_handle_domain_irq+0x34/0x50
    gic_handle_irq+0x58/0x138
    call_on_irq_stack+0x2c/0x58
    do_interrupt_handler+0x88/0x90
    el1_interrupt+0x40/0x78
    el1h_64_irq_handler+0x18/0x28
    el1h_64_irq+0x64/0x68
    trace_event_eval_update+0x114/0x420
    eval_map_work_func+0x30/0x40
    process_one_work+0x298/0x6b0
    worker_thread+0x54/0x408
    kthread+0x118/0x128
    ret_from_fork+0x10/0x20
   brbe: implementation found on cpu 1
   brbe: implementation found on cpu 2
   brbe: implementation found on cpu 3
Quoted 20 lines
drivers/perf/arm_pmu_platform.c +10 -0
+		if (err)
+			return err;
+	}
+	return err;
+}
+
 static void armpmu_free_irqs(struct arm_pmu *armpmu)
 {
 	int cpu;
@@ -229,6 +259,10 @@ int arm_pmu_device_probe(struct platform_device *pdev,
 	if (ret)
 		goto out_free_irqs;
 
+	ret = armpmu_request_brbe(pmu);
+	if (ret)
+		goto out_free_irqs;
+
 	ret = armpmu_register(pmu);
 	if (ret) {
 		dev_err(dev, "failed to register PMU devices!\n");
#10 lore raw

Re: [PATCH V3 4/7] driver/perf/arm_pmu_platform: Add support for BRBE attributes detection

Quoted 68 lines · Anshuman Khandual
Quoted 55 lines · James Clark
Quoted 53 lines · James Clark
Quoted 39 lines · Anshuman Khandual

This adds arm pmu infrastrure to probe BRBE implementation's attributes via driver exported callbacks later. The actual BRBE feature detection will be added by the driver itself.

CPU specific BRBE entries, cycle count, format support gets detected during PMU init. This information gets saved in per-cpu struct pmu_hw_events which later helps in operating BRBE during a perf event context.

---
 drivers/perf/arm_pmu_platform.c | 34 +++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
drivers/perf/arm_pmu_platform.c +14 -0
index 933b96e243b8..acdc445081aa 100644
@@ -172,6 +172,36 @@ static int armpmu_request_irqs(struct arm_pmu *armpmu)
 	return err;
 }
 
+static void arm_brbe_probe_cpu(void *info)
+{
+	struct pmu_hw_events *hw_events;
+	struct arm_pmu *armpmu = info;
+
+	/*
+	 * Return from here, if BRBE driver has not been
+	 * implemented for this PMU. This helps prevent
+	 * kernel crash later when brbe_probe() will be
+	 * called on the PMU.
+	 */
+	if (!armpmu->brbe_probe)
+		return;
+

drivers/perf/arm_pmu_platform.c static int armpmu_request_irqs(struct arm_pmu *armpmu) · line 198

Quoted 10 lines
drivers/perf/arm_pmu_platform.c +10 -0
+	hw_events = per_cpu_ptr(armpmu->hw_events, smp_processor_id());
+	armpmu->brbe_probe(hw_events);
+}
+
+static int armpmu_request_brbe(struct arm_pmu *armpmu)
+{
+	int cpu, err = 0;
+
+	for_each_cpu(cpu, &armpmu->supported_cpus) {
+		err = smp_call_function_single(cpu, arm_brbe_probe_cpu, armpmu, 1);

Hi Anshuman,

Quoted 3 lines
Quoted 2 lines

I have LOCKDEP on and the patchset applied to perf/core (82aad7ff7) on
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git and I get

Can you confirm if this is currently the correct place to apply this to?

This series applied on v6.0-rc5 after the perf ABI changes, both in kernel and in user space tools.

Quoted 3 lines

I'm only getting 0 length branch stacks now. Seems like it could be something to do with the layout of perf samples because I know that was done in separate commits:

Right, might be.

Quoted 8 lines
Quoted 6 lines

sudo ./perf record -j any_call -- ls
./perf report -D | grep "branch stack"
... branch stack: nr:0
... branch stack: nr:0
... branch stack: nr:0
... branch stack: nr:0

I am planning to respin the series on 6.1-rc1 next week which should solve these multiple moving parts problem

There are some recent changes which require PMU driver to set data.sample_flags indicating what kind of records are being filled in there. Here are the commits

a9a931e2666878343 ("perf: Use sample_flags for branch stack") 3aac580d5cc3001ca ("perf: Add sample_flags to indicate the PMU-filled sample data")

Following fix solves the problem for BRBE driver.

arch/arm64/kernel/perf_event.c +1 -0
index 98e9a615d3cb..85a3aaefc0fb 100644
@@ -877,6 +877,7 @@ static irqreturn_t armv8pmu_handle_irq(struct arm_pmu *cpu_pmu)
                if (has_branch_stack(event)) {
                        cpu_pmu->brbe_read(cpuc, event);
                        data.br_stack = &cpuc->branches->brbe_stack;
+                       data.sample_flags |= PERF_SAMPLE_BRANCH_STACK;
                        cpu_pmu->brbe_reset(cpuc);
                }
 
#11 lore raw

Re: [PATCH V3 4/7] driver/perf/arm_pmu_platform: Add support for BRBE attributes detection

In reply to James Clark
Quoted 115 lines · James Clark
Quoted 39 lines · Anshuman Khandual

This adds arm pmu infrastrure to probe BRBE implementation's attributes via driver exported callbacks later. The actual BRBE feature detection will be added by the driver itself.

CPU specific BRBE entries, cycle count, format support gets detected during PMU init. This information gets saved in per-cpu struct pmu_hw_events which later helps in operating BRBE during a perf event context.

---
 drivers/perf/arm_pmu_platform.c | 34 +++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
drivers/perf/arm_pmu_platform.c +14 -0
index 933b96e243b8..acdc445081aa 100644
@@ -172,6 +172,36 @@ static int armpmu_request_irqs(struct arm_pmu *armpmu)
 	return err;
 }
 
+static void arm_brbe_probe_cpu(void *info)
+{
+	struct pmu_hw_events *hw_events;
+	struct arm_pmu *armpmu = info;
+
+	/*
+	 * Return from here, if BRBE driver has not been
+	 * implemented for this PMU. This helps prevent
+	 * kernel crash later when brbe_probe() will be
+	 * called on the PMU.
+	 */
+	if (!armpmu->brbe_probe)
+		return;
+

drivers/perf/arm_pmu_platform.c static int armpmu_request_irqs(struct arm_pmu *armpmu) · line 198

Quoted 10 lines
drivers/perf/arm_pmu_platform.c +10 -0
+	hw_events = per_cpu_ptr(armpmu->hw_events, smp_processor_id());
+	armpmu->brbe_probe(hw_events);
+}
+
+static int armpmu_request_brbe(struct arm_pmu *armpmu)
+{
+	int cpu, err = 0;
+
+	for_each_cpu(cpu, &armpmu->supported_cpus) {
+		err = smp_call_function_single(cpu, arm_brbe_probe_cpu, armpmu, 1);

Hi Anshuman,

I have LOCKDEP on and the patchset applied to perf/core (82aad7ff7) on
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git and I get this:

armv8-pmu pmu: hw perfevents: no interrupt-affinity property, guessing.
brbe: implementation found on cpu 0

   =============================
   [ BUG: Invalid wait context ]
   6.0.0-rc7 #38 Not tainted
   -----------------------------
   kworker/u8:0/9 is trying to lock:
   ffff000800855898 (&port_lock_key){....}-{3:3}, at:
pl011_console_write+0x148/0x240
   other info that might help us debug this:
   context-{2:2}
   5 locks held by kworker/u8:0/9:
    #0: ffff00080032a138 ((wq_completion)eval_map_wq){+.+.}-{0:0}, at:
process_one_work+0x200/0x6b0
    #1: ffff80000807bde0
((work_completion)(&eval_map_work)){+.+.}-{0:0}, at:
process_one_work+0x200/0x6b0
    #2: ffff80000aa3db70 (trace_event_sem){+.+.}-{4:4}, at:
trace_event_eval_update+0x28/0x420
    #3: ffff80000a9afe58 (console_lock){+.+.}-{0:0}, at:
vprintk_emit+0x130/0x380
    #4: ffff80000a9aff78 (console_owner){-...}-{0:0}, at:
console_emit_next_record.constprop.0+0x128/0x338
   stack backtrace:
   CPU: 0 PID: 9 Comm: kworker/u8:0 Not tainted 6.0.0-rc7 #38
   Hardware name: Foundation-v8A (DT)
   Workqueue: eval_map_wq eval_map_work_func
   Call trace:
    dump_backtrace+0x114/0x120
    show_stack+0x20/0x58
    dump_stack_lvl+0x9c/0xd8
    dump_stack+0x18/0x34
    __lock_acquire+0x17cc/0x1920
    lock_acquire+0x138/0x3b8
    _raw_spin_lock+0x58/0x70
    pl011_console_write+0x148/0x240
    console_emit_next_record.constprop.0+0x194/0x338
    console_unlock+0x18c/0x208
    vprintk_emit+0x24c/0x380
    vprintk_default+0x40/0x50
    vprintk+0xd4/0xf0
    _printk+0x68/0x90
    arm64_pmu_brbe_probe+0x10c/0x128
    armv8pmu_brbe_probe+0x18/0x28
    arm_brbe_probe_cpu+0x44/0x58
    __flush_smp_call_function_queue+0x1d0/0x440
    generic_smp_call_function_single_interrupt+0x20/0x78
    ipi_handler+0x98/0x368
    handle_percpu_devid_irq+0xc0/0x3a8
    generic_handle_domain_irq+0x34/0x50
    gic_handle_irq+0x58/0x138
    call_on_irq_stack+0x2c/0x58
    do_interrupt_handler+0x88/0x90
    el1_interrupt+0x40/0x78
    el1h_64_irq_handler+0x18/0x28
    el1h_64_irq+0x64/0x68
    trace_event_eval_update+0x114/0x420
Quoted 8 lines
    eval_map_work_func+0x30/0x40
    process_one_work+0x298/0x6b0
    worker_thread+0x54/0x408
    kthread+0x118/0x128
    ret_from_fork+0x10/0x20
   brbe: implementation found on cpu 1
   brbe: implementation found on cpu 2
   brbe: implementation found on cpu 3

The LOCKDEP warnings are because of pr_warn/pr_info in arm64_pmu_brbe_probe() which gets called from smp_call_function_single() context. I will drop these prints, instead probably capture them in struct pmu_hw_events and display in the caller itself.

#12 lore raw

PATCH v3 5/7 arm64/perf: Drive BRBE from perf event states

In reply to Anshuman Khandual

Branch stack sampling rides along the normal perf event and all the branch records get captured during the PMU interrupt. This just changes perf event handling on the arm64 platform to accommodate required BRBE operations that will enable branch stack sampling support.

Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
---
 arch/arm64/kernel/perf_event.c |  6 +++++
 drivers/perf/arm_pmu.c         | 40 ++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
arch/arm64/kernel/perf_event.c +6 -0
index e7013699171f..6793b25c3f21 100644
@@ -874,6 +874,12 @@ static irqreturn_t armv8pmu_handle_irq(struct arm_pmu *cpu_pmu)
 		if (!armpmu_event_set_period(event))
 			continue;
 
+		if (has_branch_stack(event)) {
+			cpu_pmu->brbe_read(cpuc, event);
+			data.br_stack = &cpuc->branches->brbe_stack;
+			cpu_pmu->brbe_reset(cpuc);
+		}
+
 		/*
 		 * Perf event overflow will queue the processing of the event as
 		 * an irq_work which will be taken care of in the handling of
drivers/perf/arm_pmu.c +40 -0
index 16fda9a1229f..93b36933124f 100644
@@ -271,12 +271,22 @@ armpmu_stop(struct perf_event *event, int flags)
 {
 	struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
 	struct hw_perf_event *hwc = &event->hw;
+	struct pmu_hw_events *hw_events = this_cpu_ptr(armpmu->hw_events);
 
 	/*
 	 * ARM pmu always has to update the counter, so ignore
 	 * PERF_EF_UPDATE, see comments in armpmu_start().
 	 */
 	if (!(hwc->state & PERF_HES_STOPPED)) {
+		if (has_branch_stack(event)) {
+			WARN_ON_ONCE(!hw_events->brbe_users);
+			hw_events->brbe_users--;
+			if (!hw_events->brbe_users) {
+				hw_events->brbe_context = NULL;
+				armpmu->brbe_disable(hw_events);
+			}
+		}
+
 		armpmu->disable(event);
 		armpmu_event_update(event);
 		hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE;
@@ -287,6 +297,7 @@ static void armpmu_start(struct perf_event *event, int flags)
 {
 	struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
 	struct hw_perf_event *hwc = &event->hw;
+	struct pmu_hw_events *hw_events = this_cpu_ptr(armpmu->hw_events);
 
 	/*
 	 * ARM pmu always has to reprogram the period, so ignore
@@ -304,6 +315,14 @@ static void armpmu_start(struct perf_event *event, int flags)
 	 * happened since disabling.
 	 */
 	armpmu_event_set_period(event);
+	if (has_branch_stack(event)) {
+		if (event->ctx->task && hw_events->brbe_context != event->ctx) {
+			armpmu->brbe_reset(hw_events);
+			hw_events->brbe_context = event->ctx;
+		}
+		armpmu->brbe_enable(hw_events);
+		hw_events->brbe_users++;
+	}
 	armpmu->enable(event);
 }
 
@@ -349,6 +368,10 @@ armpmu_add(struct perf_event *event, int flags)
 	hw_events->events[idx] = event;
 
 	hwc->state = PERF_HES_STOPPED | PERF_HES_UPTODATE;
+
+	if (has_branch_stack(event))
+		armpmu->brbe_filter(hw_events, event);
+
 	if (flags & PERF_EF_START)
 		armpmu_start(event, PERF_EF_RELOAD);
 
@@ -443,6 +466,7 @@ __hw_perf_event_init(struct perf_event *event)
 {
 	struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
 	struct hw_perf_event *hwc = &event->hw;
+	struct pmu_hw_events *hw_events = this_cpu_ptr(armpmu->hw_events);
 	int mapping;
 
 	hwc->flags = 0;
@@ -492,6 +516,9 @@ __hw_perf_event_init(struct perf_event *event)
 		local64_set(&hwc->period_left, hwc->sample_period);
 	}
 
+	if (has_branch_stack(event))
+		armpmu->brbe_filter(hw_events, event);
+
 	return validate_group(event);
 }
 
@@ -520,6 +547,18 @@ static int armpmu_event_init(struct perf_event *event)
 	return __hw_perf_event_init(event);
 }
 
+static void armpmu_sched_task(struct perf_event_context *ctx, bool sched_in)
+{
+	struct arm_pmu *armpmu = to_arm_pmu(ctx->pmu);
+	struct pmu_hw_events *hw_events = this_cpu_ptr(armpmu->hw_events);
+
+	if (!hw_events->brbe_users)
+		return;
+
+	if (sched_in)
+		armpmu->brbe_reset(hw_events);
+}
+
 static void armpmu_enable(struct pmu *pmu)
 {
 	struct arm_pmu *armpmu = to_arm_pmu(pmu);
@@ -877,6 +916,7 @@ static struct arm_pmu *__armpmu_alloc(gfp_t flags)
 	}
 
 	pmu->pmu = (struct pmu) {
+		.sched_task	= armpmu_sched_task,
 		.pmu_enable	= armpmu_enable,
 		.pmu_disable	= armpmu_disable,
 		.event_init	= armpmu_event_init,
Signature · 1 line
2.25.1
#13 lore raw

PATCH v3 6/7 arm64/perf: Add BRBE driver

In reply to Anshuman Khandual

This adds a BRBE driver which implements all the required helper functions for struct arm_pmu. Following functions are defined by this driver which will configure, enable, capture, reset and disable BRBE buffer HW as and when requested via perf branch stack sampling framework.

- arm64_pmu_brbe_filter()
- arm64_pmu_brbe_enable()
- arm64_pmu_brbe_disable()
- arm64_pmu_brbe_read()
- arm64_pmu_brbe_probe()
- arm64_pmu_brbe_reset()
- arm64_pmu_brbe_supported()

Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
---
 arch/arm64/kernel/perf_event.c |   8 +-
 drivers/perf/Kconfig           |  11 +
 drivers/perf/Makefile          |   1 +
 drivers/perf/arm_pmu_brbe.c    | 447 +++++++++++++++++++++++++++++++++
 drivers/perf/arm_pmu_brbe.h    | 259 +++++++++++++++++++
 include/linux/perf/arm_pmu.h   |  20 ++
 6 files changed, 745 insertions(+), 1 deletion(-)
 create mode 100644 drivers/perf/arm_pmu_brbe.c
 create mode 100644 drivers/perf/arm_pmu_brbe.h
arch/arm64/kernel/perf_event.c +7 -1
index 6793b25c3f21..6917f9b100e4 100644
@@ -1033,31 +1033,37 @@ static int armv8pmu_filter_match(struct perf_event *event)
 
 static void armv8pmu_brbe_filter(struct pmu_hw_events *hw_event, struct perf_event *event)
 {
+	arm64_pmu_brbe_filter(hw_event, event);
 }
 
 static void armv8pmu_brbe_enable(struct pmu_hw_events *hw_event)
 {
+	arm64_pmu_brbe_enable(hw_event);
 }
 
 static void armv8pmu_brbe_disable(struct pmu_hw_events *hw_event)
 {
+	arm64_pmu_brbe_disable(hw_event);
 }
 
 static void armv8pmu_brbe_read(struct pmu_hw_events *hw_event, struct perf_event *event)
 {
+	arm64_pmu_brbe_read(hw_event, event);
 }
 
 static void armv8pmu_brbe_probe(struct pmu_hw_events *hw_event)
 {
+	arm64_pmu_brbe_probe(hw_event);
 }
 
 static void armv8pmu_brbe_reset(struct pmu_hw_events *hw_event)
 {
+	arm64_pmu_brbe_reset(hw_event);
 }
 
 static bool armv8pmu_brbe_supported(struct perf_event *event)
 {
-	return false;
+	return arm64_pmu_brbe_supported(event);
 }
 
 static void armv8pmu_reset(void *info)
drivers/perf/Kconfig +11 -0
index 1e2d69453771..9fa34a1d3a23 100644
@@ -183,6 +183,17 @@ config APPLE_M1_CPU_PMU
 	  Provides support for the non-architectural CPU PMUs present on
 	  the Apple M1 SoCs and derivatives.
 
+config ARM_BRBE_PMU
+	tristate "Enable support for Branch Record Buffer Extension (BRBE)"
+	depends on ARM64 && ARM_PMU
+	default y
+	help
+	  Enable perf support for Branch Record Buffer Extension (BRBE) which
+	  records all branches taken in an execution path. This supports some
+	  branch types and privilege based filtering. It captured additional
+	  relevant information such as cycle count, misprediction and branch
+	  type, branch privilege level etc.
+
 source "drivers/perf/hisilicon/Kconfig"
 
 config MARVELL_CN10K_DDR_PMU
drivers/perf/Makefile +1 -0
index 57a279c61df5..b81fc134d95f 100644
@@ -20,3 +20,4 @@ obj-$(CONFIG_ARM_DMC620_PMU) += arm_dmc620_pmu.o
 obj-$(CONFIG_MARVELL_CN10K_TAD_PMU) += marvell_cn10k_tad_pmu.o
 obj-$(CONFIG_MARVELL_CN10K_DDR_PMU) += marvell_cn10k_ddr_pmu.o
 obj-$(CONFIG_APPLE_M1_CPU_PMU) += apple_m1_cpu_pmu.o
+obj-$(CONFIG_ARM_BRBE_PMU) += arm_pmu_brbe.o
drivers/perf/arm_pmu_brbe.c new file +447 -0

447 lines not on this page · raw message

drivers/perf/arm_pmu_brbe.h new file +259 -0

259 lines not on this page · raw message

include/linux/perf/arm_pmu.h +20 -0
index ffce43ceb670..3cd94d401f98 100644
@@ -166,6 +166,26 @@ struct arm_pmu {
 	unsigned long acpi_cpuid;
 };
 
+#ifdef CONFIG_ARM_BRBE_PMU
+void arm64_pmu_brbe_filter(struct pmu_hw_events *hw_events, struct perf_event *event);
+void arm64_pmu_brbe_read(struct pmu_hw_events *cpuc, struct perf_event *event);
+void arm64_pmu_brbe_disable(struct pmu_hw_events *cpuc);
+void arm64_pmu_brbe_enable(struct pmu_hw_events *cpuc);
+void arm64_pmu_brbe_probe(struct pmu_hw_events *cpuc);
+void arm64_pmu_brbe_reset(struct pmu_hw_events *cpuc);
+bool arm64_pmu_brbe_supported(struct perf_event *event);
+#else
+static inline void arm64_pmu_brbe_filter(struct pmu_hw_events *hw_events, struct perf_event *event)
+{
+}
+static inline void arm64_pmu_brbe_read(struct pmu_hw_events *cpuc, struct perf_event *event) { }
+static inline void arm64_pmu_brbe_disable(struct pmu_hw_events *cpuc) { }
+static inline void arm64_pmu_brbe_enable(struct pmu_hw_events *cpuc) { }
+static inline void arm64_pmu_brbe_probe(struct pmu_hw_events *cpuc) { }
+static inline void arm64_pmu_brbe_reset(struct pmu_hw_events *cpuc) { }
+static inline bool arm64_pmu_brbe_supported(struct perf_event *event) {return false; }
+#endif
+
 #define to_arm_pmu(p) (container_of(p, struct arm_pmu, pmu))
 
 u64 armpmu_event_update(struct perf_event *event);
Signature · 1 line
2.25.1
#14 lore raw

PATCH v3 7/7 arm64/perf: Enable branch stack sampling

In reply to Anshuman Khandual

Now that all the required pieces are already in place, just enable the perf branch stack sampling support on arm64 platform, by removing the gate which blocks it in armpmu_event_init().

---
 drivers/perf/arm_pmu.c | 32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)
drivers/perf/arm_pmu.c +29 -3
index 93b36933124f..2a9b988b53c2 100644
@@ -537,9 +537,35 @@ static int armpmu_event_init(struct perf_event *event)
 		!cpumask_test_cpu(event->cpu, &armpmu->supported_cpus))
 		return -ENOENT;
 
-	/* does not support taken branch sampling */
-	if (has_branch_stack(event))
-		return -EOPNOTSUPP;
+	if (has_branch_stack(event)) {
+		/*
+		 * BRBE support is absent. Select CONFIG_ARM_BRBE_PMU
+		 * in the config, before branch stack sampling events
+		 * can be requested.
+		 */
+		if (!IS_ENABLED(CONFIG_ARM_BRBE_PMU)) {
+			pr_warn_once("BRBE is disabled, select CONFIG_ARM_BRBE_PMU\n");
+			return -EOPNOTSUPP;
+		}
+
+		if (event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_KERNEL) {
+			if (!perfmon_capable()) {
+				pr_warn_once("does not have permission for kernel branch filter\n");
+				return -EPERM;
+			}
+		}
+
+		/*
+		 * Branch stack sampling event can not be supported in
+		 * case either the required driver itself is absent or
+		 * BRBE buffer, is not supported. Besides checking for
+		 * the callback prevents a crash in case it's absent.
+		 */
+		if (!armpmu->brbe_supported || !armpmu->brbe_supported(event)) {
+			pr_warn_once("BRBE is not supported\n");
+			return -EOPNOTSUPP;
+		}
+	}
 
 	if (armpmu->map_event(event) == -ENOENT)
 		return -ENOENT;
Signature · 1 line
2.25.1
#15 lore raw
Quoted 28 lines · Anshuman Khandual

Now that all the required pieces are already in place, just enable the perf branch stack sampling support on arm64 platform, by removing the gate which blocks it in armpmu_event_init().

---
 drivers/perf/arm_pmu.c | 32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)
drivers/perf/arm_pmu.c +3 -3
index 93b36933124f..2a9b988b53c2 100644
@@ -537,9 +537,35 @@ static int armpmu_event_init(struct perf_event *event)
 		!cpumask_test_cpu(event->cpu, &armpmu->supported_cpus))
 		return -ENOENT;
 
-	/* does not support taken branch sampling */
-	if (has_branch_stack(event))
-		return -EOPNOTSUPP;
+	if (has_branch_stack(event)) {
+		/*
+		 * BRBE support is absent. Select CONFIG_ARM_BRBE_PMU

drivers/perf/arm_pmu.c static int armpmu_event_init(struct perf_event *event) · line 552

Quoted 10 lines
drivers/perf/arm_pmu.c +10 -0
+		 * in the config, before branch stack sampling events
+		 * can be requested.
+		 */
+		if (!IS_ENABLED(CONFIG_ARM_BRBE_PMU)) {
+			pr_warn_once("BRBE is disabled, select CONFIG_ARM_BRBE_PMU\n");
+			return -EOPNOTSUPP;
+		}
+
+		if (event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_KERNEL) {
+			if (!perfmon_capable()) {

I'm still getting different behaviour compared to x86 when using perf_event_paranoid because of this perfmon_capable() call here.

drivers/perf/arm_pmu.c static int armpmu_event_init(struct perf_event *event) · line 553

Quoted 1 line
drivers/perf/arm_pmu.c +1 -0
+				pr_warn_once("does not have permission for kernel branch filter\n");

Also I was under the impression that this should be more like a KERN_INFO loglevel rather than a KERN_WARNING. It's more like expected behavior rather than unexpected behavior and as far as I know anyone who sees something in dmesg might think something has gone wrong and try to follow it up. It is quite a useful message but I remember getting a review like this before and it made sense to me.

Quoted 18 lines
drivers/perf/arm_pmu.c +15 -0
+				return -EPERM;
+			}
+		}
+
+		/*
+		 * Branch stack sampling event can not be supported in
+		 * case either the required driver itself is absent or
+		 * BRBE buffer, is not supported. Besides checking for
+		 * the callback prevents a crash in case it's absent.
+		 */
+		if (!armpmu->brbe_supported || !armpmu->brbe_supported(event)) {
+			pr_warn_once("BRBE is not supported\n");
+			return -EOPNOTSUPP;
+		}
+	}
 
 	if (armpmu->map_event(event) == -ENOENT)
 		return -ENOENT;
#16 lore raw
Quoted 36 lines · James Clark
Quoted 33 lines · Anshuman Khandual

Now that all the required pieces are already in place, just enable the perf branch stack sampling support on arm64 platform, by removing the gate which blocks it in armpmu_event_init().

---
  drivers/perf/arm_pmu.c | 32 +++++++++++++++++++++++++++++---
  1 file changed, 29 insertions(+), 3 deletions(-)
drivers/perf/arm_pmu.c +8 -3
index 93b36933124f..2a9b988b53c2 100644
@@ -537,9 +537,35 @@ static int armpmu_event_init(struct perf_event *event)
  		!cpumask_test_cpu(event->cpu, &armpmu->supported_cpus))
  		return -ENOENT;
 
-	/* does not support taken branch sampling */
-	if (has_branch_stack(event))
-		return -EOPNOTSUPP;
+	if (has_branch_stack(event)) {
+		/*
+		 * BRBE support is absent. Select CONFIG_ARM_BRBE_PMU
+		 * in the config, before branch stack sampling events
+		 * can be requested.
+		 */
+		if (!IS_ENABLED(CONFIG_ARM_BRBE_PMU)) {
+			pr_warn_once("BRBE is disabled, select CONFIG_ARM_BRBE_PMU\n");
Quoted 8 lines

drivers/perf/arm_pmu.c static int armpmu_event_init(struct perf_event *event) · line 552

Quoted 5 lines
drivers/perf/arm_pmu.c +5 -0
+			return -EOPNOTSUPP;
+		}
+
+		if (event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_KERNEL) {
+			if (!perfmon_capable()) {

I'm still getting different behaviour compared to x86 when using perf_event_paranoid because of this perfmon_capable() call here.

Given the generic events framework already checks this for any privileged branch samples (i.e., for both KERNEL and HV), the individual drivers must not add additional restrictions.

Quoted 9 lines

drivers/perf/arm_pmu.c static int armpmu_event_init(struct perf_event *event) · line 553

Quoted 2 lines
drivers/perf/arm_pmu.c +1 -0
+				pr_warn_once("does not have permission for kernel branch filter\n");

Also I was under the impression that this should be more like a KERN_INFO loglevel rather than a KERN_WARNING. It's more like expected behavior rather than unexpected behavior and as far as I know anyone who sees something in dmesg might think something has gone wrong and try to follow it up. It is quite a useful message but I remember getting a review like this before and it made sense to me.

+1

Suzuki

#17 lore raw
Quoted 45 lines · Suzuki K Poulose
Quoted 35 lines · James Clark
Quoted 32 lines · Anshuman Khandual

Now that all the required pieces are already in place, just enable the perf branch stack sampling support on arm64 platform, by removing the gate which blocks it in armpmu_event_init().

---
  drivers/perf/arm_pmu.c | 32 +++++++++++++++++++++++++++++---

  1 file changed, 29 insertions(+), 3 deletions(-)
drivers/perf/arm_pmu.c +0 -0
index 93b36933124f..2a9b988b53c2 100644
@@ -537,9 +537,35 @@ static int armpmu_event_init(struct perf_event *event)
           !cpumask_test_cpu(event->cpu, &armpmu->supported_cpus))
           return -ENOENT;

  -    /* does not support taken branch sampling */
-    if (has_branch_stack(event))
-        return -EOPNOTSUPP;
+    if (has_branch_stack(event)) {
+        /*
+         * BRBE support is absent. Select CONFIG_ARM_BRBE_PMU
+         * in the config, before branch stack sampling events
+         * can be requested.
+         */
+        if (!IS_ENABLED(CONFIG_ARM_BRBE_PMU)) {
+            pr_warn_once("BRBE is disabled, select CONFIG_ARM_BRBE_PMU\n");

Quoted 8 lines
Quoted 5 lines

+            return -EOPNOTSUPP;
+        }
+
+        if (event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_KERNEL) { +            if (!perfmon_capable()) {

I'm still getting different behaviour compared to x86 when using perf_event_paranoid because of this perfmon_capable() call here.

Quoted 3 lines

Given the generic events framework already checks this for any privileged branch samples (i.e., for both KERNEL and HV), the individual drivers must not add additional restrictions.

Okay, will drop perfmon_capable() check here along with the warning.

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

+                pr_warn_once("does not have permission for kernel branch filter\n");

Quoted 8 lines
Quoted 6 lines

Also I was under the impression that this should be more like a KERN_INFO loglevel rather than a KERN_WARNING. It's more like expected behavior rather than unexpected behavior and as far as I know anyone who sees something in dmesg might think something has gone wrong and try to follow it up. It is quite a useful message but I remember getting a review like this before and it made sense to me.

+1

Sure, will change remaining pr_warn_once() prints as pr_info() instead.