libperf and arm64 userspace counter access support
Subsystems (5)
v5 cover libperf and arm64 userspace counter access support
Hi all,
Another version of libperf and arm64 userspace counter access support. This originally resurrected Raphael's series[1] to enable userspace counter access on arm64. My previous versions are here[2][3][4][5]. A git branch is here[6].
Changes in v5:
- Limit enabling/disabling access to CPUs associated with the PMU (supported_cpus) and with the mm_struct matching current->active_mm. The x86 method of using mm_cpumask doesn't work for arm64 as it is not updated.
- Only set cap_user_rdpmc if event is on current cpu. See patch 2.
- Create an mmap for every event in an evsel. This results in some changes to the libperf mmap API from the last version.
- Rebase to v5.11-rc2
Changes in v4:
- Dropped 'arm64: pmu: Add hook to handle pmu-related undefined instructions'. The onus is on userspace to pin itself to a homogeneous subset of CPUs and avoid any aborts on heterogeneous systems, so the hook is not needed.
- Make perf_evsel__mmap() take pages rather than bytes for size
- Fix building arm64 heterogeneous test.
Changes in v3:
- Dropped removing x86 rdpmc test until libperf tests can run via 'perf test'
- Added verbose prints for tests
- Split adding perf_evsel__mmap() to separate patch
The following changes to the arm64 support have been made compared to Raphael's last version:
The major change is support for heterogeneous systems with some restrictions. Specifically, userspace must pin itself to like CPUs, open a specific PMU by type, and use h/w specific events. The tests have been reworked to demonstrate this.
Chained events are not supported. The problem with supporting chained events was there's no way to distinguish between a chained event and a native 64-bit counter. We could add some flag, but do self monitoring processes really need that? Native 64-bit counters are supported if the PMU h/w has support. As there's already an explicit ABI to request 64-bit counters, userspace can request 64-bit counters and if user access is not enabled, then it must retry with 32-bit counters.
Prior versions broke the build on arm32 (surprisingly never caught by 0-day). As a result, event_mapped and event_unmapped implementations have been moved into the arm64 code.
There was a bug in that pmc_width was not set in the user page. The tests now check for this.
The documentation has been converted to rST. I've added sections on chained events and heterogeneous.
The tests have been expanded to test the cycle counter access.
Rob
[1] https://lore.kernel.org/r/[email protected]/
[2] https://lore.kernel.org/r/[email protected]/
[3] https://lore.kernel.org/r/[email protected]/
[4] https://lore.kernel.org/r/[email protected]/
[5] https://lore.kernel.org/r/[email protected]/
[6] git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git user-perf-event-v5
Raphael Gault (3):
arm64: pmu: Add function implementation to update event index in
userpage
arm64: perf: Enable PMU counter direct access for perf event
Documentation: arm64: Document PMU counters access from userspace
Rob Herring (6):
tools/include: Add an initial math64.h
libperf: Add evsel mmap support
libperf: tests: Add support for verbose printing
libperf: Add support for user space counter access
libperf: Add arm64 support to perf_mmap__read_self()
perf: arm64: Add test for userspace counter access on heterogeneous
systems
Documentation/arm64/index.rst | 1 + .../arm64/perf_counter_user_access.rst | 56 ++++++ arch/arm64/include/asm/mmu.h | 5 + arch/arm64/include/asm/mmu_context.h | 2 + arch/arm64/include/asm/perf_event.h | 14 ++ arch/arm64/kernel/perf_event.c | 68 +++++++ include/linux/perf/arm_pmu.h | 2 + tools/include/linux/math64.h | 75 +++++++ tools/lib/perf/Documentation/libperf.txt | 2 + tools/lib/perf/evsel.c | 50 ++++- tools/lib/perf/include/internal/evsel.h | 2 + tools/lib/perf/include/internal/mmap.h | 3 + tools/lib/perf/include/internal/tests.h | 32 +++ tools/lib/perf/include/perf/evsel.h | 2 + tools/lib/perf/libperf.map | 2 + tools/lib/perf/mmap.c | 186 ++++++++++++++++++ tools/lib/perf/tests/Makefile | 6 +- tools/lib/perf/tests/test-evsel.c | 65 ++++++ tools/perf/arch/arm64/include/arch-tests.h | 7 + tools/perf/arch/arm64/tests/Build | 1 + tools/perf/arch/arm64/tests/arch-tests.c | 4 + tools/perf/arch/arm64/tests/user-events.c | 170 ++++++++++++++++ 22 files changed, 751 insertions(+), 4 deletions(-) create mode 100644 Documentation/arm64/perf_counter_user_access.rst create mode 100644 tools/include/linux/math64.h create mode 100644 tools/perf/arch/arm64/tests/user-events.c
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v5 1/9 arm64: pmu: Add function implementation to update event index in userpage
From: Raphael Gault <[email protected]>
In order to be able to access the counter directly for userspace, we need to provide the index of the counter using the userpage. We thus need to override the event_idx function to retrieve and convert the perf_event index to armv8 hardware index.
Since the arm_pmu driver can be used by any implementation, even if not armv8, two components play a role into making sure the behaviour is correct and consistent with the PMU capabilities:
* the ARMPMU_EL0_RD_CNTR flag which denotes the capability to access counter from userspace.
* the event_idx call back, which is implemented and initialized by the PMU implementation: if no callback is provided, the default behaviour applies, returning 0 as index value.
--- arch/arm64/kernel/perf_event.c | 21 +++++++++++++++++++++ include/linux/perf/arm_pmu.h | 2 ++ 2 files changed, 23 insertions(+)
arch/arm64/kernel/perf_event.c +21 -0
@@ -873,6 +873,22 @@ static void armv8pmu_clear_event_idx(struct pmu_hw_events *cpuc,clear_bit (idx -1 , cpuc->used_mask); } +static int armv8pmu_access_event_idx (struct perf_event *event) +{ +if (!(event->hw.flags &ARMPMU_EL0_RD_CNTR )) +return 0 ; + +/* +* We remap the cycle counter index to 32 to +* match the offset applied to the rest of +* the counter indices. +*/ +if (event->hw.idx ==ARMV8_IDX_CYCLE_COUNTER ) +return 32 ; + +return event->hw.idx; +} +/* * Add an event filter to a given event. */
@@ -969,6 +985,9 @@ static int __armv8_pmuv3_map_event(struct perf_event *event,if (armv8pmu_event_is_64bit (event)) event->hw.flags |=ARMPMU_EVT_64BIT ; +if (!armv8pmu_event_is_chained (event)) + event->hw.flags |=ARMPMU_EL0_RD_CNTR ; +/* Only expose micro/arch events supported by this PMU */ if ((hw_event_id >0 ) && (hw_event_id <ARMV8_PMUV3_MAX_COMMON_EVENTS ) &&test_bit (hw_event_id, armpmu->pmceid_bitmap)) {
@@ -1100,6 +1119,8 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name, cpu_pmu->set_event_filter = armv8pmu_set_event_filter; cpu_pmu->filter_match = armv8pmu_filter_match; + cpu_pmu->pmu.event_idx = armv8pmu_access_event_idx; + cpu_pmu->name = name; cpu_pmu->map_event = map_event; cpu_pmu->attr_groups[ARMPMU_ATTR_GROUP_EVENTS] = events ?
include/linux/perf/arm_pmu.h +2 -0
@@ -26,6 +26,8 @@*/ /* Event uses a 64bit counter */ #define ARMPMU_EVT_64BIT 1 +/* Allow access to hardware counter from userspace */ +#define ARMPMU_EL0_RD_CNTR 2#define HW_OP_UNSUPPORTED 0xFFFF#define C (_x) PERF_COUNT_HW_CACHE_##_x
Signature · 7 lines
2.27.0 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v5 2/9 arm64: perf: Enable PMU counter direct access for perf event
In reply to Rob HerringFrom: Raphael Gault <[email protected]>
Keep track of event opened with direct access to the hardware counters and modify permissions while they are open.
The strategy used here is the same which x86 uses: every time an event is mapped, the permissions are set if required. The atomic field added in the mm_context helps keep track of the different event opened and de-activate the permissions when all are unmapped.
We also need to update the permissions in the context switch code so that tasks keep the right permissions.
---
I'm not completely sure if using current->active_mm in an IPI is okay? It seems to work in my testing.
Peter Z says (event->oncpu == smp_processor_id()) in the user page update is always true, but my testing says otherwise[1].
v5:
- Only set cap_user_rdpmc if event is on current cpu
- Limit enabling/disabling access to CPUs associated with the PMU (supported_cpus) and with the mm_struct matching current->active_mm.
v2:
- Move mapped/unmapped into arm64 code. Fixes arm32.
- Rebase on cap_user_time_short changes
Changes from Raphael's v4:
- Drop homogeneous check
- Disable access for chained counters
- Set pmc_width in user page
[1] https://lore.kernel.org/lkml/CAL_JsqK+eKef5NaVnBfARCjRE3MYhfBfe54F9YHKbsTnWqLmLw@mail.gmail.com/
--- arch/arm64/include/asm/mmu.h | 5 +++ arch/arm64/include/asm/mmu_context.h | 2 ++ arch/arm64/include/asm/perf_event.h | 14 +++++++++ arch/arm64/kernel/perf_event.c | 47 ++++++++++++++++++++++++++++ 4 files changed, 68 insertions(+)
arch/arm64/include/asm/mmu.h +5 -0
@@ -18,6 +18,11 @@ typedef struct { atomic64_t id; + /* + * non-zero if userspace have access to hardware + * counters directly. + */ + atomic_t pmu_direct_access; #ifdef CONFIG_COMPAT void *sigpage; #endif
arch/arm64/include/asm/mmu_context.h +2 -0
@@ -21,6 +21,7 @@#include <asm/proc-fns.h> #include <asm-generic/mm_hooks.h> #include <asm/cputype.h> +#include <asm/perf_event.h> #include <asm/sysreg.h> #include <asm/tlbflush.h>
@@ -231,6 +232,7 @@ static inline void __switch_mm(struct mm_struct *next) }check_and_switch_context (next); +perf_switch_user_access (next); } static inline void
arch/arm64/include/asm/perf_event.h +14 -0
@@ -8,6 +8,7 @@#include <asm/stack_pointer.h> #include <asm/ptrace.h> +#include <linux/mm_types.h> #define ARMV8_PMU_MAX_COUNTERS 32#define ARMV8_PMU_COUNTER_MASK (ARMV8_PMU_MAX_COUNTERS - 1)
@@ -254,4 +255,17 @@ extern unsigned long perf_misc_flags(struct pt_regs *regs); (regs)->pstate = PSR_MODE_EL1h; \ } +static inline void perf_switch_user_access (struct mm_struct *mm) +{ +if (!IS_ENABLED (CONFIG_PERF_EVENTS )) +return ; + +if (atomic_read (&mm->context.pmu_direct_access)) { +write_sysreg (ARMV8_PMU_USERENR_ER |ARMV8_PMU_USERENR_CR , + pmuserenr_el0); + }else { +write_sysreg (0 , pmuserenr_el0); + } +} +#endif
arch/arm64/kernel/perf_event.c +47 -0
65 lines not on this page · raw message
Signature · 7 lines
2.27.0 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v5 3/9 tools/include: Add an initial math64.h
In reply to Rob HerringAdd an initial math64.h similar to linux/math64.h with functions mul_u64_u64_div64() and mul_u64_u32_shr(). This isn't a direct copy of include/linux/math64.h as that doesn't define mul_u64_u64_div64().
Implementation was written by Peter Zilkstra based on linux/math64.h and div64.h[1]. The original implementation was not optimal on arm64 as __int128 division is not optimal with a call out to __udivti3, so I dropped the __int128 variant of mul_u64_u64_div64().
[1] https://lore.kernel.org/lkml/[email protected]/
--- tools/include/linux/math64.h | 75 ++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 tools/include/linux/math64.h
tools/include/linux/math64.h new file +75 -0
@@ -0,0 +1,75 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_MATH64_H +#define _LINUX_MATH64_H + +#include <linux/types.h> + +#ifdef __x86_64__ +static inline u64 mul_u64_u64_div64 (u64 a,u64 b,u64 c) +{ +u64 q; + + asm ("mulq %2; divq %3" :"=a" (q) + :"a" (a),"rm" (b),"rm" (c) + :"rdx" ); + +return q; +} +#define mul_u64_u64_div64 mul_u64_u64_div64 +#endif + +#ifdef __SIZEOF_INT128__ +static inline u64 mul_u64_u32_shr (u64 a,u32 b,unsigned int shift) +{ +return (u64)(((unsigned __int128 )a * b) >> shift); +} + +#else + +#ifdef __i386__ +static inline u64 mul_u32_u32 (u32 a,u32 b) +{ +u32 high, low; + + asm ("mull %[b]" :"=a" (low),"=d" (high) + : [a]"a" (a), [b]"rm" (b) ); + +return low | ((u64 )high) <<32 ; +} +#else +static inline u64 mul_u32_u32 (u32 a,u32 b) +{ +return (u64 )a * b; +} +#endif + +static inline u64 mul_u64_u32_shr (u64 a,u32 b,unsigned int shift) +{ +u32 ah, al; +u64 ret; + + al = a; + ah = a >>32 ; + + ret =mul_u32_u32 (al, b) >> shift; +if (ah) + ret +=mul_u32_u32 (ah, b) << (32 - shift); + +return ret; +} + +#endif /* __SIZEOF_INT128__ */ + +#ifndef mul_u64_u64_div64 +static inline u64 mul_u64_u64_div64 (u64 a,u64 b,u64 c) +{ +u64 quot, rem; + + quot = a / c; + rem = a % c; + +return quot * b + (rem * b) / c; +} +#endif + +#endif /* _LINUX_MATH64_H */
Signature · 7 lines
2.27.0 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v5 4/9 libperf: Add evsel mmap support
In reply to Rob HerringIn order to support usersapce access, an event must be mmapped. While there's already mmap support for evlist, the usecase is a bit different than the self monitoring with userspace access. So let's add a new perf_evsel__mmap() function to mmap an evsel. This allows implementing userspace access as a fastpath for perf_evsel__read().
The mmapped address is returned by perf_evsel__mmap_base() which primarily for users/tests to check if userspace access is enabled.
---
v5:
- Create an mmap for every underlying event opened. Due to this, we need a different way to get the mmap ptr, so perf_evsel__mmap_base() is introduced.
v4:
- Change perf_evsel__mmap size to pages instead of bytes
v3:
- New patch split out from user access patch
--- tools/lib/perf/Documentation/libperf.txt | 2 + tools/lib/perf/evsel.c | 47 +++++++++++++++++++++++- tools/lib/perf/include/internal/evsel.h | 2 + tools/lib/perf/include/perf/evsel.h | 2 + tools/lib/perf/libperf.map | 2 + 5 files changed, 53 insertions(+), 2 deletions(-)
tools/lib/perf/Documentation/libperf.txt +2 -0
@@ -136,6 +136,8 @@ SYNOPSIS struct perf_thread_map *threads); void perf_evsel__close(struct perf_evsel *evsel); void perf_evsel__close_cpu(struct perf_evsel *evsel, int cpu); + int perf_evsel__mmap(struct perf_evsel *evsel, int pages); + void *perf_evsel__mmap_base(struct perf_evsel *evsel, int cpu, int thread); int perf_evsel__read(struct perf_evsel *evsel, int cpu, int thread, struct perf_counts_values *count); int perf_evsel__enable(struct perf_evsel *evsel);
tools/lib/perf/evsel.c +45 -2
72 lines not on this page · raw message
tools/lib/perf/include/internal/evsel.h +2 -0
@@ -9,6 +9,7 @@struct perf_cpu_map ;struct perf_thread_map ; +struct perf_mmap ;struct xyarray ;/*
@@ -40,6 +41,7 @@ struct perf_evsel {struct perf_cpu_map *cpus;struct perf_cpu_map *own_cpus;struct perf_thread_map *threads; +struct perf_mmap *mmap;struct xyarray *fd;struct xyarray *sample_id;u64 *id;
tools/lib/perf/include/perf/evsel.h +2 -0
@@ -27,6 +27,8 @@ LIBPERF_API int perf_evsel__open(struct perf_evsel *evsel, struct perf_cpu_map * struct perf_thread_map *threads); LIBPERF_API void perf_evsel__close(struct perf_evsel *evsel); LIBPERF_API void perf_evsel__close_cpu(struct perf_evsel *evsel, int cpu); +LIBPERF_API int perf_evsel__mmap(struct perf_evsel *evsel, int pages); +LIBPERF_API void *perf_evsel__mmap_base(struct perf_evsel *evsel, int cpu, int thread); LIBPERF_API int perf_evsel__read(struct perf_evsel *evsel, int cpu, int thread,struct perf_counts_values *count); LIBPERF_API int perf_evsel__enable(struct perf_evsel *evsel);
tools/lib/perf/libperf.map +2 -0
@@ -23,6 +23,8 @@ LIBPERF_0.0.1 { perf_evsel__disable; perf_evsel__open; perf_evsel__close; + perf_evsel__mmap; + perf_evsel__mmap_base; perf_evsel__read; perf_evsel__cpus; perf_evsel__threads;
Signature · 7 lines
2.27.0 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v5 4/9] libperf: Add evsel mmap support
Quoted 58 lines · Rob Herring
In order to support usersapce access, an event must be mmapped. While there's already mmap support for evlist, the usecase is a bit different than the self monitoring with userspace access. So let's add a new
perf_evsel__mmap()function to mmap an evsel. This allows implementing userspace access as a fastpath forperf_evsel__read().The mmapped address is returned by
perf_evsel__mmap_base()which primarily for users/tests to check if userspace access is enabled.Signed-off-by Rob Herring <[email protected]>---
v5:
- Create an mmap for every underlying event opened. Due to this, we need a different way to get the mmap ptr, soperf_evsel__mmap_base()is introduced.
v4:
- Changeperf_evsel__mmapsize to pages instead of bytes
v3:
- New patch split out from user access patch--- tools/lib/perf/Documentation/libperf.txt | 2 + tools/lib/perf/evsel.c | 47 +++++++++++++++++++++++- tools/lib/perf/include/internal/evsel.h | 2 + tools/lib/perf/include/perf/evsel.h | 2 + tools/lib/perf/libperf.map | 2 + 5 files changed, 53 insertions(+), 2 deletions(-)tools/lib/perf/Documentation/libperf.txt +2
-0@@ -136,6 +136,8 @@ SYNOPSIS struct perf_thread_map *threads); void perf_evsel__close(struct perf_evsel *evsel); void perf_evsel__close_cpu(struct perf_evsel *evsel, int cpu); + int perf_evsel__mmap(struct perf_evsel *evsel, int pages); + void *perf_evsel__mmap_base(struct perf_evsel *evsel, int cpu, int thread); int perf_evsel__read(struct perf_evsel *evsel, int cpu, int thread, struct perf_counts_values *count); int perf_evsel__enable(struct perf_evsel *evsel);tools/lib/perf/evsel.c +2
-0@@ -11,10 +11,12 @@#include <stdlib.h> #include <internal/xyarray.h> #include <internal/cpumap.h> +#include <internal/mmap.h> #include <internal/threadmap.h> #include <internal/lib.h> #include <linux/string.h> #include <sys/ioctl.h> +#include <sys/mman.h> void perf_evsel__init (struct perf_evsel *evsel,struct perf_event_attr *attr) {
tools/lib/perf/evsel.c void perf_evsel__delete(struct perf_evsel *evsel) · line 45
Quoted 8 lines
tools/lib/perf/evsel.c +4
-1@@ -37,11 +39,17 @@ void perf_evsel__delete(struct perf_evsel *evsel)free (evsel); } -#define FD (e, x, y) (*(int *) xyarray__entry(e->fd, x, y)) +struct evsel_fd { +int fd; +struct perf_mmap mmap; +};
nice shortcut ;-) but 'struct perf_mmap' is too big for that
I think it's better to add new 'evsel::mmap' xyarray to hold it, add perf_evsel__alloc_mmap to allocate it and call it from perf_evsel__mmap the same way as we callperf_evsel__alloc_fd from perf_evsel__open
thanks,
jirka
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v5 5/9 libperf: tests: Add support for verbose printing
In reply to Rob HerringAdd __T_VERBOSE() so tests can add verbose output. The verbose output is enabled with the '-v' command line option.
---
v5:
- Pass verbose flag to static tests
- Fix getopt loop with unsigned char (arm64)
v3:
- New patch
--- tools/lib/perf/include/internal/tests.h | 32 +++++++++++++++++++++++++ tools/lib/perf/tests/Makefile | 6 +++-- 2 files changed, 36 insertions(+), 2 deletions(-)
tools/lib/perf/include/internal/tests.h +32 -0
@@ -3,11 +3,32 @@#define __LIBPERF_INTERNAL_TESTS_H#include <stdio.h> +#include <unistd.h> int tests_failed; +int tests_verbose; + +static inline int get_verbose (char **argv,int argc) +{ +int c; +int verbose =0 ; + +while ((c =getopt (argc, argv,"v" )) !=-1 ) { +switch (c) + { +case 'v' : + verbose =1 ; +break ; +default : +break ; + } + } +return verbose; +}#define __T_START \ do { \ + tests_verbose = get_verbose(argv, argc); \ fprintf(stdout, "- running %s...", __FILE__); \ fflush(NULL); \ tests_failed = 0; \
@@ -30,4 +51,15 @@ do { } \ } while (0) +#define __T_VERBOSE (...) \ +do { \ + if (tests_verbose) { \ + if (tests_verbose == 1) { \ + fputc('\n', stderr); \ + tests_verbose++; \ + } \ + fprintf(stderr, ##__VA_ARGS__); \ + } \ +} while (0) +#endif /* __LIBPERF_INTERNAL_TESTS_H */
tools/lib/perf/tests/Makefile +4 -2
@@ -5,6 +5,8 @@ TESTS = test-cpumap test-threadmap test-evlist test-evsel TESTS_SO := $(addsuffix -so,$(TESTS)) TESTS_A := $(addsuffix -a,$(TESTS)) +TEST_ARGS := $(if $(V),-v) + # Set compile option CFLAGS ifdef EXTRA_CFLAGS CFLAGS := $(EXTRA_CFLAGS)
@@ -28,9 +30,9 @@ all: $(TESTS_A) $(TESTS_SO) run: @echo "running static:" - @for i in $(TESTS_A); do ./$$i; done + @for i in $(TESTS_A); do ./$$i $(TEST_ARGS); done @echo "running dynamic:" - @for i in $(TESTS_SO); do LD_LIBRARY_PATH=../ ./$$i; done + @for i in $(TESTS_SO); do LD_LIBRARY_PATH=../ ./$$i $(TEST_ARGS); done clean: $(call QUIET_CLEAN, tests)$(RM) $(TESTS_A) $(TESTS_SO)
Signature · 7 lines
2.27.0 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v5 6/9 libperf: Add support for user space counter access
In reply to Rob Herringx86 and arm64 can both support direct access of event counters in userspace. The access sequence is less than trivial and currently exists in perf test code (tools/perf/arch/x86/tests/rdpmc.c) with copies in projects such as PAPI and libpfm4.
In order to support usersapce access, an event must be mmapped first with perf_evsel__mmap(). Then subsequent calls to perf_evsel__read() will use the fast path (assuming the arch supports it).
---
v5:
- Make raw count s64 instead of u64 so that counter width shifting works
- Adapt to mmap changes
v4:
- Update perf_evsel__mmap size to pages
v3:
- Split out perf_evsel__mmap() to separate patch
squash! libperf: Add support for user space counter access
--- tools/lib/perf/evsel.c | 3 + tools/lib/perf/include/internal/mmap.h | 3 + tools/lib/perf/mmap.c | 88 ++++++++++++++++++++++++++ tools/lib/perf/tests/test-evsel.c | 65 +++++++++++++++++++ 4 files changed, 159 insertions(+)
tools/lib/perf/evsel.c +3 -0
@@ -234,6 +234,9 @@ int perf_evsel__read(struct perf_evsel *evsel, int cpu, int thread,if (FD (evsel, cpu, thread) <0 )return -EINVAL ; +if (!perf_mmap__read_self (MMAP (evsel, cpu, thread), count)) +return 0 ; +if (readn (FD (evsel, cpu, thread), count->values, size) <=0 )return -errno;
tools/lib/perf/include/internal/mmap.h +3 -0
@@ -11,6 +11,7 @@#define PERF_SAMPLE_MAX_SIZE (1 << 16)struct perf_mmap ; +struct perf_counts_values ;typedef void (*libperf_unmap_cb_t )(struct perf_mmap *map);
@@ -52,4 +53,6 @@ void perf_mmap__put(struct perf_mmap *map);u64 perf_mmap__read_head (struct perf_mmap *map); +int perf_mmap__read_self (struct perf_mmap *map,struct perf_counts_values *count); +#endif /* __LIBPERF_INTERNAL_MMAP_H */
tools/lib/perf/mmap.c +88 -0
100 lines not on this page · raw message
tools/lib/perf/tests/test-evsel.c +65 -0
77 lines not on this page · raw message
Signature · 1 line
2.27.0
PATCH v5 7/9 libperf: Add arm64 support to perf_mmap__read_self()
In reply to Rob HerringAdd the arm64 variants for read_perf_counter() and read_timestamp(). Unfortunately the counter number is encoded into the instruction, so the code is a bit verbose to enumerate all possible counters.
--- tools/lib/perf/mmap.c | 98 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+)
tools/lib/perf/mmap.c +98 -0
@@ -13,6 +13,7 @@#include <internal/lib.h> #include <linux/kernel.h> #include <linux/math64.h> +#include <linux/stringify.h> #include "internal.h" void perf_mmap__init(struct perf_mmap *map, struct perf_mmap *prev,
@@ -294,6 +295,103 @@ static u64 read_timestamp(void)return low | ((u64 )high) <<32 ; } +#elif defined(__aarch64__) +#define read_sysreg (r) ({ \ + u64 __val; \ + asm volatile("mrs %0, " __stringify(r) : "=r" (__val)); \ + __val; \ +}) + +static u64 read_pmccntr (void ) +{ +return read_sysreg (pmccntr_el0); +} + +#define PMEVCNTR_READ (idx) \ + static u64 read_pmevcntr_##idx(void) { \ + return read_sysreg(pmevcntr##idx##_el0); \ + } + +PMEVCNTR_READ (0 ); +PMEVCNTR_READ (1 ); +PMEVCNTR_READ (2 ); +PMEVCNTR_READ (3 ); +PMEVCNTR_READ (4 ); +PMEVCNTR_READ (5 ); +PMEVCNTR_READ (6 ); +PMEVCNTR_READ (7 ); +PMEVCNTR_READ (8 ); +PMEVCNTR_READ (9 ); +PMEVCNTR_READ (10 ); +PMEVCNTR_READ (11 ); +PMEVCNTR_READ (12 ); +PMEVCNTR_READ (13 ); +PMEVCNTR_READ (14 ); +PMEVCNTR_READ (15 ); +PMEVCNTR_READ (16 ); +PMEVCNTR_READ (17 ); +PMEVCNTR_READ (18 ); +PMEVCNTR_READ (19 ); +PMEVCNTR_READ (20 ); +PMEVCNTR_READ (21 ); +PMEVCNTR_READ (22 ); +PMEVCNTR_READ (23 ); +PMEVCNTR_READ (24 ); +PMEVCNTR_READ (25 ); +PMEVCNTR_READ (26 ); +PMEVCNTR_READ (27 ); +PMEVCNTR_READ (28 ); +PMEVCNTR_READ (29 ); +PMEVCNTR_READ (30 ); + +/* +* Read a value direct from PMEVCNTR<idx> +*/ +static u64 read_perf_counter (unsigned int counter) +{ +static u64 (*const read_f[])(void ) = { + read_pmevcntr_0, + read_pmevcntr_1, + read_pmevcntr_2, + read_pmevcntr_3, + read_pmevcntr_4, + read_pmevcntr_5, + read_pmevcntr_6, + read_pmevcntr_7, + read_pmevcntr_8, + read_pmevcntr_9, + read_pmevcntr_10, + read_pmevcntr_11, + read_pmevcntr_13, + read_pmevcntr_12, + read_pmevcntr_14, + read_pmevcntr_15, + read_pmevcntr_16, + read_pmevcntr_17, + read_pmevcntr_18, + read_pmevcntr_19, + read_pmevcntr_20, + read_pmevcntr_21, + read_pmevcntr_22, + read_pmevcntr_23, + read_pmevcntr_24, + read_pmevcntr_25, + read_pmevcntr_26, + read_pmevcntr_27, + read_pmevcntr_28, + read_pmevcntr_29, + read_pmevcntr_30, + read_pmccntr + }; + +if (counter <ARRAY_SIZE (read_f)) +return (read_f[counter])(); + +return 0 ; +} + +static u64 read_timestamp (void ) {return read_sysreg (cntvct_el0); } +#else static u64 read_perf_counter (unsigned int counter) {return 0 ; }static u64 read_timestamp (void ) {return 0 ; }
Signature · 1 line
2.27.0
PATCH v5 8/9 perf: arm64: Add test for userspace counter access on heterogeneous systems
In reply to Rob HerringUserspace counter access only works on heterogeneous systems with some restrictions. The userspace process must be pinned to a homogeneous subset of CPUs and must open the corresponding PMU for those CPUs. This commit adds a test implementing these requirements.
---
v5:
- Adapt to libperf mmap API changes
v4:
- Update perf_evsel__mmap params
v2:
- Drop all but heterogeneous test as others covered by libperf tests
- Rework to use libperf
--- tools/perf/arch/arm64/include/arch-tests.h | 7 + tools/perf/arch/arm64/tests/Build | 1 + tools/perf/arch/arm64/tests/arch-tests.c | 4 + tools/perf/arch/arm64/tests/user-events.c | 170 +++++++++++++++++++++ 4 files changed, 182 insertions(+) create mode 100644 tools/perf/arch/arm64/tests/user-events.c
tools/perf/arch/arm64/include/arch-tests.h +7 -0
@@ -2,11 +2,18 @@#ifndef ARCH_TESTS_H #define ARCH_TESTS_H +#include <linux/compiler.h> +#ifdef HAVE_DWARF_UNWIND_SUPPORT struct thread ;struct perf_sample ; +int test__arch_unwind_sample (struct perf_sample *sample, +struct thread *thread);#endif extern struct test arch_tests[]; +int test__rd_pinned (struct test __maybe_unused *test, +int __maybe_unused subtest); +#endif
tools/perf/arch/arm64/tests/Build +1 -0
@@ -1,4 +1,5 @@ perf-y += regs_load.o perf-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o +perf-y += user-events.o perf-y += arch-tests.o
tools/perf/arch/arm64/tests/arch-tests.c +4 -0
@@ -10,6 +10,10 @@ struct test arch_tests[] = { .func = test__dwarf_unwind, },#endif + { + .desc = "Pinned CPU user counter access", + .func = test__rd_pinned, + }, { .func = NULL, },
tools/perf/arch/arm64/tests/user-events.c new file +170 -0
170 lines not on this page · raw message
Signature · 1 line
2.27.0
PATCH v5 9/9 Documentation: arm64: Document PMU counters access from userspace
In reply to Rob HerringFrom: Raphael Gault <[email protected]>
Add a documentation file to describe the access to the pmu hardware counters from userspace
---
v2:
- Update links to test examples
Changes from Raphael's v4:
- Convert to rSt
- Update chained event status
- Add section for heterogeneous systems
--- Documentation/arm64/index.rst | 1 + .../arm64/perf_counter_user_access.rst | 56 +++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 Documentation/arm64/perf_counter_user_access.rst
Documentation/arm64/index.rst +1 -0
@@ -18,6 +18,7 @@ ARM64 Architecture memory memory-tagging-extension perf + perf_counter_user_access pointer-authentication silicon-errata sve
Documentation/arm64/perf_counter_user_access.rst new file +56 -0
56 lines not on this page · raw message
Signature · 1 line
2.27.0
Quoted 6 lines · Rob Herring
Hi all,
Another version of libperf and arm64 userspace counter access support. This originally resurrected Raphael's series[1] to enable userspace counter access on arm64. My previous versions are here[2][3][4][5]. A git branch is here[6].
Ping.
Will, Mark, comments on the Arm bits?
Quoted 104 lines
Changes in v5:
- Limit enabling/disabling access to CPUs associated with the PMU (supported_cpus) and with themm_structmatchingcurrent->active_mm. The x86 method of usingmm_cpumaskdoesn't work for arm64 as it is not updated.
- Only setcap_user_rdpmcif event is on current cpu. See patch 2.
- Create an mmap for every event in an evsel. This results in some changes to the libperf mmap API from the last version.
- Rebase to v5.11-rc2Changes in v4:
- Dropped 'arm64: pmu: Add hook to handle pmu-related undefined instructions'. The onus is on userspace to pin itself to a homogeneous subset of CPUs and avoid any aborts on heterogeneous systems, so the hook is not needed.
- Makeperf_evsel__mmap()take pages rather than bytes for size
- Fix building arm64 heterogeneous test.Changes in v3:
- Dropped removing x86 rdpmc test until libperf tests can run via 'perf test'
- Added verbose prints for tests
- Split addingperf_evsel__mmap()to separate patchThe following changes to the arm64 support have been made compared to Raphael's last version:
The major change is support for heterogeneous systems with some restrictions. Specifically, userspace must pin itself to like CPUs, open a specific PMU by type, and use h/w specific events. The tests have been reworked to demonstrate this.
Chained events are not supported. The problem with supporting chained events was there's no way to distinguish between a chained event and a native 64-bit counter. We could add some flag, but do self monitoring processes really need that? Native 64-bit counters are supported if the PMU h/w has support. As there's already an explicit ABI to request 64-bit counters, userspace can request 64-bit counters and if user access is not enabled, then it must retry with 32-bit counters.
Prior versions broke the build on arm32 (surprisingly never caught by 0-day). As a result,
event_mappedandevent_unmappedimplementations have been moved into the arm64 code.There was a bug in that
pmc_widthwas not set in the user page. The tests now check for this.The documentation has been converted to rST. I've added sections on chained events and heterogeneous.
The tests have been expanded to test the cycle counter access.
Rob
[1] https://lore.kernel.org/r/[email protected]/
[2] https://lore.kernel.org/r/[email protected]/
[3] https://lore.kernel.org/r/[email protected]/
[4] https://lore.kernel.org/r/[email protected]/
[5] https://lore.kernel.org/r/[email protected]/
[6]git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.gituser-perf-event-v5Raphael Gault (3):
arm64: pmu: Add function implementation to update event index in
userpage
arm64: perf: Enable PMU counter direct access for perf event
Documentation: arm64: Document PMU counters access from userspaceRob Herring (6):
tools/include: Add an initialmath64.h
libperf: Add evsel mmap support
libperf: tests: Add support for verbose printing
libperf: Add support for user space counter access
libperf: Add arm64 support toperf_mmap__read_self()
perf: arm64: Add test for userspace counter access on heterogeneous
systemsDocumentation/arm64/index.rst | 1 + .../arm64/perf_counter_user_access.rst | 56 ++++++ arch/arm64/include/asm/mmu.h | 5 + arch/arm64/include/asm/mmu_context.h | 2 + arch/arm64/include/asm/perf_event.h | 14 ++ arch/arm64/kernel/perf_event.c | 68 +++++++ include/linux/perf/arm_pmu.h | 2 + tools/include/linux/math64.h | 75 +++++++ tools/lib/perf/Documentation/libperf.txt | 2 + tools/lib/perf/evsel.c | 50 ++++- tools/lib/perf/include/internal/evsel.h | 2 + tools/lib/perf/include/internal/mmap.h | 3 + tools/lib/perf/include/internal/tests.h | 32 +++ tools/lib/perf/include/perf/evsel.h | 2 + tools/lib/perf/libperf.map | 2 + tools/lib/perf/mmap.c | 186 ++++++++++++++++++ tools/lib/perf/tests/Makefile | 6 +- tools/lib/perf/tests/test-evsel.c | 65 ++++++ tools/perf/arch/arm64/include/arch-tests.h | 7 + tools/perf/arch/arm64/tests/Build | 1 + tools/perf/arch/arm64/tests/arch-tests.c | 4 + tools/perf/arch/arm64/tests/user-events.c | 170 ++++++++++++++++ 22 files changed, 751 insertions(+), 4 deletions(-) create mode 100644 Documentation/arm64/perf_counter_user_access.rst create mode 100644 tools/include/linux/math64.h create mode 100644 tools/perf/arch/arm64/tests/user-events.c
Quoted 2 lines
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel