arm64: Enable access to pmu registers by user-space
Subsystems (5)
cover arm64: Enable access to pmu registers by user-space
This is resurrecting Raphael's series[1] to enable userspace counter access. The following changes have been made from the last version:
The major change is support for heterogeneous systems with some restrictions. Specifically, userspace must pin itself to like CPUs, open a specific PMU by type, and use h/w specific events. The tests have been reworked to demonstrate this.
Chained events are not supported. The problem with supporting chained events was there's no way to distinguish between a chained event and a native 64-bit counter. We could add some flag, but do self monitoring processes really need that? Native 64-bit counters are supported if the PMU h/w has support. As there's already an explicit ABI to request 64-bit counters, userspace can request 64-bit counters and if user access is not enabled, then it must retry with 32-bit counters.
There was a bug in that pmc_width was not set in the user page. The tests now check for this.
The documentation has been converted to rST. I've added sections on chained events and heterogeneous.
The tests have been expanded to test the cycle counter access.
Rob
[1] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Raphael Gault (5):
perf: arm64: Add tests to check userspace access to hardware counters
arm64: pmu: Add hook to handle pmu-related undefined instructions
arm64: pmu: Add function implementation to update event index in
userpage
arm64: perf: Enable pmu counter direct access for perf event on armv8
Documentation: arm64: Document PMU counters access from userspace
Documentation/arm64/index.rst | 1 + .../arm64/perf_counter_user_access.rst | 52 ++ arch/arm64/include/asm/mmu.h | 6 + arch/arm64/include/asm/mmu_context.h | 2 + arch/arm64/include/asm/perf_event.h | 14 + arch/arm64/kernel/cpufeature.c | 4 +- arch/arm64/kernel/perf_event.c | 80 +++ drivers/perf/arm_pmu.c | 38 ++ include/linux/perf/arm_pmu.h | 2 + tools/perf/arch/arm64/include/arch-tests.h | 11 + tools/perf/arch/arm64/tests/Build | 1 + tools/perf/arch/arm64/tests/arch-tests.c | 12 + tools/perf/arch/arm64/tests/user-events.c | 454 ++++++++++++++++++ 13 files changed, 675 insertions(+), 2 deletions(-) create mode 100644 Documentation/arm64/perf_counter_user_access.rst create mode 100644 tools/perf/arch/arm64/tests/user-events.c
--
2.25.1
PATCH 1/5 perf: arm64: Add tests to check userspace access to hardware counters
From: Raphael Gault <[email protected]>
These tests test PMU registers access from userspace. It uses perf_event_mmap_page() to retrieve the counter index and access the underlying register.
For heterogeneous systems, userspace is expected to pin itself to like CPUs and open the corresponding PMU. Test test__rd_pinned() demonstrates this.
---
Changes:
- Rebase to v5.8 and update includes
- Add cycle counter test
- Add test for heterogeneous systems
- check pmc_width
- Tests check access first instead of getting garbage data
--- tools/perf/arch/arm64/include/arch-tests.h | 11 + tools/perf/arch/arm64/tests/Build | 1 + tools/perf/arch/arm64/tests/arch-tests.c | 12 + tools/perf/arch/arm64/tests/user-events.c | 454 +++++++++++++++++++++ 4 files changed, 478 insertions(+) create mode 100644 tools/perf/arch/arm64/tests/user-events.c
tools/perf/arch/arm64/include/arch-tests.h +11 -0
@@ -2,11 +2,22 @@#ifndef ARCH_TESTS_H #define ARCH_TESTS_H +#include <linux/compiler.h> +#ifdef HAVE_DWARF_UNWIND_SUPPORT struct thread ;struct perf_sample ; +int test__arch_unwind_sample (struct perf_sample *sample, +struct thread *thread);#endif extern struct test arch_tests[]; +int test__rd_pmevcntr (struct test *test__maybe_unused , +int subtest__maybe_unused ); +int test__rd_pmccntr (struct test *test__maybe_unused , +int subtest__maybe_unused ); +int test__rd_pinned (struct test __maybe_unused *test, +int __maybe_unused subtest); +#endif
tools/perf/arch/arm64/tests/Build +1 -0
@@ -1,4 +1,5 @@ perf-y += regs_load.o perf-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o +perf-y += user-events.o perf-y += arch-tests.o
tools/perf/arch/arm64/tests/arch-tests.c +12 -0
@@ -10,6 +10,18 @@ struct test arch_tests[] = { .func = test__dwarf_unwind, },#endif + { + .desc = "User event counter access", + .func = test__rd_pmevcntr, + }, + { + .desc = "User cycle counter access", + .func = test__rd_pmccntr, + }, + { + .desc = "Pinned CPU user counter access", + .func = test__rd_pinned, + }, { .func = NULL, },
tools/perf/arch/arm64/tests/user-events.c new file +454 -0
454 lines not on this page · raw message
--
2.25.1
PATCH 2/5 arm64: pmu: Add hook to handle pmu-related undefined instructions
In reply to Rob HerringFrom: Raphael Gault <[email protected]>
This patch introduces a protection for the userspace processes which are trying to access the registers from the pmu registers on a big.LITTLE environment. It introduces a hook to handle undefined instructions.
The goal here is to prevent the process to be interrupted by a signal when the error is caused by the task being scheduled while accessing a counter, causing the counter access to be invalid. As we are not able to know efficiently the number of counters available physically on both pmu in that context we consider that any faulting access to a counter which is architecturally correct should not cause a SIGILL signal if the permissions are set accordingly.
This commit also modifies the mask of the mrs_hook declared in arch/arm64/kernel/cpufeatures.c which emulates only feature register access. This is necessary because this hook's mask was too large and thus masking any mrs instruction, even if not related to the emulated registers which made the pmu emulation inefficient.
--- arch/arm64/kernel/cpufeature.c | 4 +-- arch/arm64/kernel/perf_event.c | 55 ++++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+), 2 deletions(-)
arch/arm64/kernel/cpufeature.c +2 -2
@@ -2744,8 +2744,8 @@ static int emulate_mrs(struct pt_regs *regs, u32 insn) }static struct undef_hook mrs_hook = { - .instr_mask =0xfff00000 , - .instr_val =0xd5300000 , + .instr_mask =0xffff0000 , + .instr_val =0xd5380000 , .pstate_mask =PSR_AA32_MODE_MASK , .pstate_val = PSR_MODE_EL0t, .fn = emulate_mrs,
arch/arm64/kernel/perf_event.c +55 -0
70 lines not on this page · raw message
--
2.25.1
kernel test robot
build botRe: [PATCH 2/5] arm64: pmu: Add hook to handle pmu-related undefined instructions
Thank you for the patch! Perhaps something to improve:
Hi Rob,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on v5.8-rc2]
[also build test WARNING on next-20200708]
[cannot apply to tip/perf/core arm64/for-next/core arm-perf/for-next/perf linus/master v5.8-rc4 v5.8-rc3] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Rob-Herring/arm64-Enable-access-to-pmu-registers-by-user-space/20200708-045931 base: 48778464bb7d346b47157d21ffde2af6b2d39110 config: arm64-allyesconfig (attached as .config) compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
arch/arm64/kernel/perf_event.c:139:41: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_CACHE_REFILL_RD'
139 | [C(L1D)][C(OP_READ)][C(RESULT_MISS)] = ARMV8_IMPDEF_PERFCTR_L1D_CACHE_REFILL_RD,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:84:45: warning: initialized field overwritten [-Woverride-init]
84 | #define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_WR 0x41
| ^~~~
arch/arm64/kernel/perf_event.c:140:44: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_CACHE_WR'
140 | [C(L1D)][C(OP_WRITE)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_L1D_CACHE_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:84:45: note: (near initialization for 'armv8_vulcan_perf_cache_map[0][1][0]')
84 | #define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_WR 0x41
| ^~~~
arch/arm64/kernel/perf_event.c:140:44: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_CACHE_WR'
140 | [C(L1D)][C(OP_WRITE)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_L1D_CACHE_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:86:51: warning: initialized field overwritten [-Woverride-init]
86 | #define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_REFILL_WR 0x43
| ^~~~
arch/arm64/kernel/perf_event.c:141:42: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_CACHE_REFILL_WR'
141 | [C(L1D)][C(OP_WRITE)][C(RESULT_MISS)] = ARMV8_IMPDEF_PERFCTR_L1D_CACHE_REFILL_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:86:51: note: (near initialization for 'armv8_vulcan_perf_cache_map[0][1][1]')
86 | #define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_REFILL_WR 0x43
| ^~~~
arch/arm64/kernel/perf_event.c:141:42: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_CACHE_REFILL_WR'
141 | [C(L1D)][C(OP_WRITE)][C(RESULT_MISS)] = ARMV8_IMPDEF_PERFCTR_L1D_CACHE_REFILL_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:95:44: warning: initialized field overwritten [-Woverride-init]
95 | #define ARMV8_IMPDEF_PERFCTR_L1D_TLB_RD 0x4E
| ^~~~
arch/arm64/kernel/perf_event.c:143:44: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_TLB_RD'
143 | [C(DTLB)][C(OP_READ)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_L1D_TLB_RD,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:95:44: note: (near initialization for 'armv8_vulcan_perf_cache_map[3][0][0]')
95 | #define ARMV8_IMPDEF_PERFCTR_L1D_TLB_RD 0x4E
| ^~~~
arch/arm64/kernel/perf_event.c:143:44: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_TLB_RD'
143 | [C(DTLB)][C(OP_READ)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_L1D_TLB_RD,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:96:44: warning: initialized field overwritten [-Woverride-init]
96 | #define ARMV8_IMPDEF_PERFCTR_L1D_TLB_WR 0x4F
| ^~~~
arch/arm64/kernel/perf_event.c:144:45: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_TLB_WR'
144 | [C(DTLB)][C(OP_WRITE)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_L1D_TLB_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:96:44: note: (near initialization for 'armv8_vulcan_perf_cache_map[3][1][0]')
96 | #define ARMV8_IMPDEF_PERFCTR_L1D_TLB_WR 0x4F
| ^~~~
arch/arm64/kernel/perf_event.c:144:45: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_TLB_WR'
144 | [C(DTLB)][C(OP_WRITE)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_L1D_TLB_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:93:50: warning: initialized field overwritten [-Woverride-init]
93 | #define ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_RD 0x4C
| ^~~~
arch/arm64/kernel/perf_event.c:145:42: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_RD'
145 | [C(DTLB)][C(OP_READ)][C(RESULT_MISS)] = ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_RD,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:93:50: note: (near initialization for 'armv8_vulcan_perf_cache_map[3][0][1]')
93 | #define ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_RD 0x4C
| ^~~~
arch/arm64/kernel/perf_event.c:145:42: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_RD'
145 | [C(DTLB)][C(OP_READ)][C(RESULT_MISS)] = ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_RD,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:94:50: warning: initialized field overwritten [-Woverride-init]
94 | #define ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_WR 0x4D
| ^~~~
arch/arm64/kernel/perf_event.c:146:43: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_WR'
146 | [C(DTLB)][C(OP_WRITE)][C(RESULT_MISS)] = ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:94:50: note: (near initialization for 'armv8_vulcan_perf_cache_map[3][1][1]')
94 | #define ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_WR 0x4D
| ^~~~
arch/arm64/kernel/perf_event.c:146:43: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_WR'
146 | [C(DTLB)][C(OP_WRITE)][C(RESULT_MISS)] = ARMV8_IMPDEF_PERFCTR_L1D_TLB_REFILL_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:110:46: warning: initialized field overwritten [-Woverride-init]
110 | #define ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_RD 0x60
| ^~~~
arch/arm64/kernel/perf_event.c:148:44: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_RD'
148 | [C(NODE)][C(OP_READ)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_RD,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:110:46: note: (near initialization for 'armv8_vulcan_perf_cache_map[6][0][0]')
110 | #define ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_RD 0x60
| ^~~~
arch/arm64/kernel/perf_event.c:148:44: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_RD'
148 | [C(NODE)][C(OP_READ)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_RD,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:111:46: warning: initialized field overwritten [-Woverride-init]
111 | #define ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_WR 0x61
| ^~~~
arch/arm64/kernel/perf_event.c:149:45: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_WR'
149 | [C(NODE)][C(OP_WRITE)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/include/asm/perf_event.h:111:46: note: (near initialization for 'armv8_vulcan_perf_cache_map[6][1][0]')
111 | #define ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_WR 0x61
| ^~~~
arch/arm64/kernel/perf_event.c:149:45: note: in expansion of macro 'ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_WR'
149 | [C(NODE)][C(OP_WRITE)][C(RESULT_ACCESS)] = ARMV8_IMPDEF_PERFCTR_BUS_ACCESS_WR,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm64/kernel/perf_event.c: In function 'emulate_pmu':
>> arch/arm64/kernel/perf_event.c:998:6: warning: variable 'sys_reg' set but not used [-Wunused-but-set-variable]
998 | u32 sys_reg, rt;
| ^~~~~~~vim +/sys_reg +998 arch/arm64/kernel/perf_event.c
995
996 static int emulate_pmu(struct pt_regs *regs, u32 insn)
997 {
> 998 u32 sys_reg, rt;
999 u32 pmuserenr;
1000
1001 sys_reg = (u32)aarch64_insn_decode_immediate(AARCH64_INSN_IMM_16, insn) << 5;
1002 rt = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RT, insn);
1003 pmuserenr = read_sysreg(pmuserenr_el0);
1004
1005 if ((pmuserenr & (ARMV8_PMU_USERENR_ER|ARMV8_PMU_USERENR_CR)) !=
1006 (ARMV8_PMU_USERENR_ER|ARMV8_PMU_USERENR_CR))
1007 return -EINVAL;
1008
1009
1010 /*
1011 * Userspace is expected to only use this in the context of the scheme
1012 * described in the struct perf_event_mmap_page comments.
1013 *
1014 * Given that context, we can only get here if we got migrated between
1015 * getting the register index and doing the MSR read. This in turn
1016 * implies we'll fail the sequence and retry, so any value returned is
1017 * 'good', all we need is to be non-fatal.
1018 *
1019 * The choice of the value 0 is comming from the fact that when
1020 * accessing a register which is not counting events but is accessible,
1021 * we get 0.
1022 */
1023 pt_regs_write_reg(regs, rt, 0);
1024
1025 arm64_skip_faulting_instruction(regs, 4);
1026 return 0;
1027 }
1028
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/[email protected]
PATCH 3/5 arm64: pmu: Add function implementation to update event index in userpage
In reply to Rob HerringFrom: Raphael Gault <[email protected]>
In order to be able to access the counter directly for userspace, we need to provide the index of the counter using the userpage. We thus need to override the event_idx function to retrieve and convert the perf_event index to armv8 hardware index.
Since the arm_pmu driver can be used by any implementation, even if not armv8, two components play a role into making sure the behaviour is correct and consistent with the PMU capabilities:
* the ARMPMU_EL0_RD_CNTR flag which denotes the capability to access counter from userspace.
* the event_idx call back, which is implemented and initialized by the PMU implementation: if no callback is provided, the default behaviour applies, returning 0 as index value.
--- arch/arm64/kernel/perf_event.c | 21 +++++++++++++++++++++ include/linux/perf/arm_pmu.h | 2 ++ 2 files changed, 23 insertions(+)
arch/arm64/kernel/perf_event.c +21 -0
@@ -795,6 +795,22 @@ static void armv8pmu_clear_event_idx(struct pmu_hw_events *cpuc,clear_bit (idx -1 , cpuc->used_mask); } +static int armv8pmu_access_event_idx (struct perf_event *event) +{ +if (!(event->hw.flags &ARMPMU_EL0_RD_CNTR )) +return 0 ; + +/* +* We remap the cycle counter index to 32 to +* match the offset applied to the rest of +* the counter indices. +*/ +if (event->hw.idx ==ARMV8_IDX_CYCLE_COUNTER ) +return 32 ; + +return event->hw.idx; +} +/* * Add an event filter to a given event. */
@@ -891,6 +907,9 @@ static int __armv8_pmuv3_map_event(struct perf_event *event,if (armv8pmu_event_is_64bit (event)) event->hw.flags |=ARMPMU_EVT_64BIT ; +if (!armv8pmu_event_is_chained (event)) + event->hw.flags |=ARMPMU_EL0_RD_CNTR ; +/* Only expose micro/arch events supported by this PMU */ if ((hw_event_id >0 ) && (hw_event_id <ARMV8_PMUV3_MAX_COMMON_EVENTS ) &&test_bit (hw_event_id, armpmu->pmceid_bitmap)) {
@@ -1068,6 +1087,8 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name, cpu_pmu->set_event_filter = armv8pmu_set_event_filter; cpu_pmu->filter_match = armv8pmu_filter_match; + cpu_pmu->pmu.event_idx = armv8pmu_access_event_idx; + cpu_pmu->name = name; cpu_pmu->map_event = map_event; cpu_pmu->attr_groups[ARMPMU_ATTR_GROUP_EVENTS] = events ?
include/linux/perf/arm_pmu.h +2 -0
@@ -26,6 +26,8 @@*/ /* Event uses a 64bit counter */ #define ARMPMU_EVT_64BIT 1 +/* Allow access to hardware counter from userspace */ +#define ARMPMU_EL0_RD_CNTR 2#define HW_OP_UNSUPPORTED 0xFFFF#define C (_x) PERF_COUNT_HW_CACHE_##_x
--
2.25.1
PATCH 4/5 arm64: perf: Enable pmu counter direct access for perf event on armv8
In reply to Rob HerringFrom: Raphael Gault <[email protected]>
Keep track of event opened with direct access to the hardware counters and modify permissions while they are open.
The strategy used here is the same which x86 uses: everytime an event is mapped, the permissions are set if required. The atomic field added in the mm_context helps keep track of the different event opened and de-activate the permissions when all are unmapped.
We also need to update the permissions in the context switch code so that tasks keep the right permissions.
---
Changes:
- Drop homogeneous check
- Disable access for chained counters
- Set pmc_width in user page
--- arch/arm64/include/asm/mmu.h | 6 +++++ arch/arm64/include/asm/mmu_context.h | 2 ++ arch/arm64/include/asm/perf_event.h | 14 ++++++++++ arch/arm64/kernel/perf_event.c | 4 +++ drivers/perf/arm_pmu.c | 38 ++++++++++++++++++++++++++++ 5 files changed, 64 insertions(+)
arch/arm64/include/asm/mmu.h +6 -0
@@ -19,6 +19,12 @@typedef struct {atomic64_t id; + +/* +* non-zero if userspace have access to hardware +* counters directly. +*/ +atomic_t pmu_direct_access;void *vdso;unsigned long flags; }mm_context_t ;
arch/arm64/include/asm/mmu_context.h +2 -0
@@ -21,6 +21,7 @@#include <asm/proc-fns.h> #include <asm-generic/mm_hooks.h> #include <asm/cputype.h> +#include <asm/perf_event.h> #include <asm/sysreg.h> #include <asm/tlbflush.h>
@@ -226,6 +227,7 @@ static inline void __switch_mm(struct mm_struct *next) }check_and_switch_context (next, cpu); +perf_switch_user_access (next); } static inline void
arch/arm64/include/asm/perf_event.h +14 -0
@@ -8,6 +8,7 @@#include <asm/stack_pointer.h> #include <asm/ptrace.h> +#include <linux/mm_types.h> #define ARMV8_PMU_MAX_COUNTERS 32#define ARMV8_PMU_COUNTER_MASK (ARMV8_PMU_MAX_COUNTERS - 1)
@@ -224,4 +225,17 @@ extern unsigned long perf_misc_flags(struct pt_regs *regs); (regs)->pstate = PSR_MODE_EL1h; \ } +static inline void perf_switch_user_access (struct mm_struct *mm) +{ +if (!IS_ENABLED (CONFIG_PERF_EVENTS )) +return ; + +if (atomic_read (&mm->context.pmu_direct_access)) { +write_sysreg (ARMV8_PMU_USERENR_ER |ARMV8_PMU_USERENR_CR , + pmuserenr_el0); + }else { +write_sysreg (0 , pmuserenr_el0); + } +} +#endif
arch/arm64/kernel/perf_event.c +4 -0
@@ -1250,6 +1250,10 @@ void arch_perf_update_userpage(struct perf_event *event,*/ freq =arch_timer_get_rate (); userpg->cap_user_time =1 ; + userpg->cap_user_rdpmc = !!(event->hw.flags &ARMPMU_EL0_RD_CNTR ); + +if (userpg->cap_user_rdpmc) + userpg->pmc_width =armv8pmu_event_is_64bit (event) ?64 :32 ;clocks_calc_mult_shift (&userpg->time_mult, &shift, freq,NSEC_PER_SEC ,0 );
drivers/perf/arm_pmu.c +38 -0
@@ -25,6 +25,7 @@#include <linux/irqdesc.h> #include <asm/irq_regs.h> +#include <asm/mmu_context.h> static DEFINE_PER_CPU(struct arm_pmu *, cpu_armpmu); static DEFINE_PER_CPU(int, cpu_irq);
@@ -778,6 +779,41 @@ static void cpu_pmu_destroy(struct arm_pmu *cpu_pmu) &cpu_pmu->node); } +static void refresh_pmuserenr (void *mm) +{ +perf_switch_user_access (mm); +} + +static void armpmu_event_mapped (struct perf_event *event,struct mm_struct *mm) +{ +if (!(event->hw.flags &ARMPMU_EL0_RD_CNTR )) +return ; + +/* +* This function relies on not being called concurrently in two +* tasks in the same mm. Otherwise one task could observe +* pmu_direct_access > 1 and return all the way back to +* userspace with user access disabled while another task is still +* doing on_each_cpu_mask() to enable user access. +* +* For now, this can't happen because all callers hold mmap_sem +* for write. If this changes, we'll need a different solution. +*/ +lockdep_assert_held_write (&mm->mmap_lock); + +if (atomic_inc_return (&mm->context.pmu_direct_access) ==1 ) +on_each_cpu (refresh_pmuserenr, mm,1 ); +} + +static void armpmu_event_unmapped (struct perf_event *event,struct mm_struct *mm) +{ +if (!(event->hw.flags &ARMPMU_EL0_RD_CNTR )) +return ; + +if (atomic_dec_and_test (&mm->context.pmu_direct_access)) +on_each_cpu_mask (mm_cpumask (mm), refresh_pmuserenr,NULL ,1 ); +} +static struct arm_pmu *__armpmu_alloc (gfp_t flags) {struct arm_pmu *pmu;
@@ -799,6 +835,8 @@ static struct arm_pmu *__armpmu_alloc(gfp_t flags) .pmu_enable = armpmu_enable, .pmu_disable = armpmu_disable, .event_init = armpmu_event_init, + .event_mapped = armpmu_event_mapped, + .event_unmapped = armpmu_event_unmapped, .add = armpmu_add, .del = armpmu_del, .start = armpmu_start,
--
2.25.1
PATCH 5/5 Documentation: arm64: Document PMU counters access from userspace
In reply to Rob HerringFrom: Raphael Gault <[email protected]>
Add a documentation file to describe the access to the pmu hardware counters from userspace
---
Changes:
- Convert to rSt
- Update chained event status
- Add section for heterogeneous systems
--- Documentation/arm64/index.rst | 1 + .../arm64/perf_counter_user_access.rst | 52 +++++++++++++++++++ 2 files changed, 53 insertions(+) create mode 100644 Documentation/arm64/perf_counter_user_access.rst
Documentation/arm64/index.rst +1 -0
@@ -14,6 +14,7 @@ ARM64 Architecture hugetlbpage legacy_instructions memory + perf_counter_user_access pointer-authentication silicon-errata sve
Documentation/arm64/perf_counter_user_access.rst new file +52 -0
52 lines not on this page · raw message
--
2.25.1