perf: Rework event forwarding logic
Subsystems (2)
v3 cover perf: Rework event forwarding logic
Usually events are opened on the same pmu as requested by user via perf_event_attr->type argument. But certain special events are internally redirected to different pmu. Currently such pmus needs to be treated specially and thus requires some gruesome hacks.
An approach, suggested by Peter Zijlstra, to get rid of these hacks was to overwrite event attributes with new pmu's attribute values within the kernel and let perf_event_init() retry opening the event with overwritten values. This patch series implements it.
v2: https://lore.kernel.org/r/[email protected]v2->v3:
- Both uses different approach to solve the problem. v2 was using new error value -ESRCH to let perf_event_init() try opening event with all pmus, hoping that the event will get forwarded to right pmu. OTOH, v3 uses different approach as described in the description.
- v3 also fixes SW pmu to task-clock/cpu-clock pmu event forwarding.
Patches are prepared on v6.3.
Ravi Bangoria (3):
perf/core: Rework forwarding of {task|cpu}-clock events perf/ibs: Fix interface via core pmu events
perf test: Add selftest to test IBS invocation via core pmu events
arch/x86/events/amd/core.c | 2 +- arch/x86/events/amd/ibs.c | 53 ++++++++--------- arch/x86/include/asm/perf_event.h | 2 + include/linux/perf_event.h | 11 ++++ kernel/events/core.c | 79 +++++++++++-------------- tools/perf/tests/Build | 1 + tools/perf/tests/amd-ibs-via-core-pmu.c | 72 ++++++++++++++++++++++ tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/tests.h | 1 + 9 files changed, 151 insertions(+), 71 deletions(-) create mode 100644 tools/perf/tests/amd-ibs-via-core-pmu.c
Signature · 1 line
2.40.0
PATCH v3 1/3 perf/core: Rework forwarding of {task|cpu}-clock events
Currently, PERF_TYPE_SOFTWARE is treated specially since task-clock and cpu-clock events are interfaced through it but internally gets forwarded to their own pmus.
Rework this by overwriting event->attr.type in perf_swevent_init() which will cause perf_init_event() to retry with updated type and event will automatically get forwarded to right pmu. With the change, SW pmu no longer needs to be treated specially and can be included in 'pmu_idr' list.
--- include/linux/perf_event.h | 11 ++++++ kernel/events/core.c | 69 ++++++++++++++++++++------------------ 2 files changed, 47 insertions(+), 33 deletions(-)
include/linux/perf_event.h +11 -0
@@ -322,6 +322,9 @@ struct pmu {/* number of address filters this PMU can do */ unsigned int nr_addr_filters; +/* Skip creating pmu device and sysfs interface. */ +bool skip_sysfs_dev; +/* * Fully disable/enable this PMU, can be used to protect from the PMI * as well as for lazy/batch writing of the MSRs.
@@ -827,6 +830,14 @@ struct perf_event {void *security;#endif struct list_head sb_list; + +/* +* Certain events gets forwarded to another pmu internally by over- +* writing kernel copy of event->attr.type without user being aware +* of it. event->orig_type contains original 'type' requested by +* user. +*/ +__u32 orig_type;#endif /* CONFIG_PERF_EVENTS */ };
kernel/events/core.c +36 -33
166 lines not on this page · raw message
Signature · 1 line
2.40.0
Peter Zijlstra
maintainerRe: [PATCH v3 1/3] perf/core: Rework forwarding of {task|cpu}-clock events
Quoted 22 lines · Ravi Bangoria
Currently,
PERF_TYPE_SOFTWAREis treated specially since task-clock and cpu-clock events are interfaced through it but internally gets forwarded to their own pmus.Rework this by overwriting
event->attr.typeinperf_swevent_init()which will causeperf_init_event()to retry with updated type and event will automatically get forwarded to right pmu. With the change, SW pmu no longer needs to be treated specially and can be included in 'pmu_idr' list.Suggested-by Peter Zijlstra <[email protected]>Signed-off-by Ravi Bangoria <[email protected]>--- include/linux/perf_event.h | 11 ++++++ kernel/events/core.c | 69 ++++++++++++++++++++------------------ 2 files changed, 47 insertions(+), 33 deletions(-)include/linux/perf_event.h +0
-0
include/linux/perf_event.h struct pmu { · line 330
Quoted 9 lines
include/linux/perf_event.h +3
-0@@ -322,6 +322,9 @@ struct pmu {/* number of address filters this PMU can do */ unsigned int nr_addr_filters; +/* Skip creating pmu device and sysfs interface. */ +bool skip_sysfs_dev; +/* * Fully disable/enable this PMU, can be used to protect from the PMI * as well as for lazy/batch writing of the MSRs.
Does this make sense?
---
include/linux/perf_event.h +2 -3
@@ -295,6 +295,8 @@ struct perf_event_pmu_context;struct perf_output_handle ; +#define PMU_NULL_DEV ((void *)(~0)) +/** * struct pmu - generic performance monitoring unit */
@@ -322,9 +324,6 @@ struct pmu {/* number of address filters this PMU can do */ unsigned int nr_addr_filters; -/* Skip creating pmu device and sysfs interface. */ -bool skip_sysfs_dev; -/* * Fully disable/enable this PMU, can be used to protect from the PMI * as well as for lazy/batch writing of the MSRs.
kernel/events/core.c +5 -5
@@ -11113,7 +11113,7 @@ static struct pmu perf_cpu_clock = { .task_ctx_nr = perf_sw_context, .capabilities = PERF_PMU_CAP_NO_NMI, - .skip_sysfs_dev = true, + .dev = PMU_NULL_DEV, .event_init = cpu_clock_event_init, .add = cpu_clock_event_add,
@@ -11195,7 +11195,7 @@ static struct pmu perf_task_clock = { .task_ctx_nr = perf_sw_context, .capabilities = PERF_PMU_CAP_NO_NMI, - .skip_sysfs_dev = true, + .dev = PMU_NULL_DEV, .event_init = task_clock_event_init, .add = task_clock_event_add,
@@ -11442,7 +11442,7 @@ int perf_pmu_register(struct pmu *pmu, c type = ret; pmu->type = type; -if (pmu_bus_running && !pmu->skip_sysfs_dev) { +if (pmu_bus_running && !pmu->dev) { ret =pmu_dev_alloc (pmu);if (ret) goto free_idr;
@@ -11524,7 +11524,7 @@ void perf_pmu_unregister(struct pmu *pmufree_percpu (pmu->pmu_disable_count);idr_remove (&pmu_idr, pmu->type); -if (pmu_bus_running) { +if (pmu_bus_running && pmu->dev !=PMU_NULL_DEV ) {if (pmu->nr_addr_filters)device_remove_file (pmu->dev, &dev_attr_nr_addr_filters);device_del (pmu->dev);
@@ -13687,7 +13687,7 @@ static int __init perf_event_sysfs_init( goto unlock;list_for_each_entry (pmu, &pmus, entry) { -if (pmu->skip_sysfs_dev) +if (pmu->dev)continue ; ret =pmu_dev_alloc (pmu);
Re: [PATCH v3 1/3] perf/core: Rework forwarding of {task|cpu}-clock events
Quoted 4 lines
Quoted 4 lines
include/linux/perf_event.h +0
-0@@ -322,6 +322,9 @@ struct pmu {/* number of address filters this PMU can do */ unsigned int nr_addr_filters;
Quoted 8 lines
include/linux/perf_event.h struct pmu { · line 329
Quoted 6 lines
include/linux/perf_event.h +3
-0+/* Skip creating pmu device and sysfs interface. */ +bool skip_sysfs_dev; +/* * Fully disable/enable this PMU, can be used to protect from the PMI * as well as for lazy/batch writing of the MSRs. Does this make sense?
Yes! Will respin with this included.
Thanks,
Ravi
PATCH v3 2/3 perf/ibs: Fix interface via core pmu events
In reply to Ravi BangoriaAlthough, IBS pmus can be invoked via their own interface, indirect IBS invocation via core pmu events is also supported with fixed set of events: cpu-cycles:p, r076:p (same as cpu-cycles:p) and r0C1:p (micro-ops) for user convenience.
This indirect IBS invocation is broken since commit 66d258c5b048 ("perf/core: Optimize perf_init_event()"), which added RAW pmu under 'pmu_idr' list and thus if event_init() fails with RAW pmu, it started returning error instead of trying other pmus.
Forward precise events from core pmu to IBS by overwriting 'type' and 'config' in the kernel copy of perf_event_attr. Overwriting will cause perf_init_event() to retry with updated 'type' and 'config', which will automatically forward event to IBS pmu.
Without patch:
$ sudo ./perf record -C 0 -e r076:p -- sleep 1 Error: The r076:p event is not supported.
With patch:
$ sudo ./perf record -C 0 -e r076:p -- sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.341 MB perf.data (37 samples) ]
Searching for the right pmu by iterating over all pmus is no longer required since all pmus now *must* be present in the 'pmu_idr' list. So, remove linear searching code.
--- arch/x86/events/amd/core.c | 2 +- arch/x86/events/amd/ibs.c | 53 +++++++++++++++---------------- arch/x86/include/asm/perf_event.h | 2 ++ kernel/events/core.c | 10 ------ 4 files changed, 29 insertions(+), 38 deletions(-)
arch/x86/events/amd/core.c +1 -1
@@ -374,7 +374,7 @@ static int amd_pmu_hw_config(struct perf_event *event)/* pass precise event sampling to ibs: */ if (event->attr.precise_ip &&get_ibs_caps ()) -return -ENOENT ; +return forward_event_to_ibs (event);if (has_branch_stack (event) && !x86_pmu.lbr_nr)return -EOPNOTSUPP ;
arch/x86/events/amd/ibs.c +26 -27
80 lines not on this page · raw message
arch/x86/include/asm/perf_event.h +2 -0
@@ -475,8 +475,10 @@ struct pebs_xmm {#ifdef CONFIG_X86_LOCAL_APIC extern u32 get_ibs_caps (void ); +extern int forward_event_to_ibs (struct perf_event *event);#else static inline u32 get_ibs_caps (void ) {return 0 ; } +static inline int forward_event_to_ibs (struct perf_event *event) {return -ENOENT ; }#endif #ifdef CONFIG_PERF_EVENTS
kernel/events/core.c +0 -10
@@ -11648,16 +11648,6 @@ static struct pmu *perf_init_event(struct perf_event *event) goto unlock; } -list_for_each_entry_rcu (pmu, &pmus, entry,lockdep_is_held (&pmus_srcu)) { - ret =perf_try_init_event (pmu, event); -if (!ret) - goto unlock; - -if (ret != -ENOENT ) { - pmu =ERR_PTR (ret); - goto unlock; - } - } fail: pmu =ERR_PTR (-ENOENT ); unlock:
Signature · 1 line
2.40.0
Peter Zijlstra
maintainerRe: [PATCH v3 2/3] perf/ibs: Fix interface via core pmu events
Quoted 11 lines · Ravi Bangoria
kernel/events/core.c +0
-3@@ -11648,16 +11648,6 @@ static struct pmu *perf_init_event(struct perf_event *event) goto unlock; } -list_for_each_entry_rcu (pmu, &pmus, entry,lockdep_is_held (&pmus_srcu)) { - ret =perf_try_init_event (pmu, event); -if (!ret)
kernel/events/core.c static struct pmu *perf_init_event(struct perf_event *event) · line 11653
Quoted 10 lines
kernel/events/core.c +0
-7- goto unlock; - -if (ret != -ENOENT ) { - pmu =ERR_PTR (ret); - goto unlock; - } - } fail: pmu =ERR_PTR (-ENOENT ); unlock:
Would it make sense to put the above and the below cleanup in a separate patch after this?
kernel/events/core.c +13 -14
@@ -11631,25 +11631,24 @@ static struct pmu *perf_init_event(strucrcu_read_lock (); pmu =idr_find (&pmu_idr, type);rcu_read_unlock (); -if (pmu) { -if (event->attr.type != type && type !=PERF_TYPE_RAW && - !(pmu->capabilities &PERF_PMU_CAP_EXTENDED_HW_TYPE )) - goto fail; - - ret =perf_try_init_event (pmu, event); -if (ret == -ENOENT && event->attr.type != type && !extended_type) { - type = event->attr.type; - goto again; - } +if (!pmu) + goto unlock; -if (ret) - pmu =ERR_PTR (ret); + ret = -ENOENT ; +if (event->attr.type != type && type !=PERF_TYPE_RAW && + !(pmu->capabilities &PERF_PMU_CAP_EXTENDED_HW_TYPE )) + goto fail; - goto unlock; + ret =perf_try_init_event (pmu, event); +if (ret == -ENOENT && event->attr.type != type && !extended_type) { + type = event->attr.type; + goto again; } fail: - pmu =ERR_PTR (-ENOENT ); +if (ret) + pmu =ERR_PTR (ret); + unlock:srcu_read_unlock (&pmus_srcu, idx);
Re: [PATCH v3 2/3] perf/ibs: Fix interface via core pmu events
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Ravi Bangoria
kernel/events/core.c +0
-0@@ -11648,16 +11648,6 @@ static struct pmu *perf_init_event(struct perf_event *event) goto unlock; }
Quoted 16 lines
kernel/events/core.c static struct pmu *perf_init_event(struct perf_event *event) · line 11653
Quoted 13 lines
kernel/events/core.c +0
-10-list_for_each_entry_rcu (pmu, &pmus, entry,lockdep_is_held (&pmus_srcu)) { - ret =perf_try_init_event (pmu, event); -if (!ret) - goto unlock; - -if (ret != -ENOENT ) { - pmu =ERR_PTR (ret); - goto unlock; - } - } fail: pmu =ERR_PTR (-ENOENT ); unlock:Would it make sense to put the above and the below cleanup in a separate patch after this?
Sure. I'll cleanup and respin.
Thanks,
Ravi
PATCH v3 3/3 perf test: Add selftest to test IBS invocation via core pmu events
In reply to Ravi BangoriaIBS pmu can be invoked via fixed set of core pmu events with 'precise_ip' set to 1. Add a simple event open test for all these events.
Without kernel fix:
$ sudo ./perf test -vv 76 76: AMD IBS via core pmu : --- start --- test child forked, pid 6553 Using CPUID AuthenticAMD-25-1-1 type: 0x0, config: 0x0, fd: 3 - Pass type: 0x0, config: 0x1, fd: -1 - Pass type: 0x4, config: 0x76, fd: -1 - Fail type: 0x4, config: 0xc1, fd: -1 - Fail type: 0x4, config: 0x12, fd: -1 - Pass test child finished with -1 ---- end ---- AMD IBS via core pmu: FAILED!
With kernel fix:
$ sudo ./perf test -vv 76 76: AMD IBS via core pmu : --- start --- test child forked, pid 7526 Using CPUID AuthenticAMD-25-1-1 type: 0x0, config: 0x0, fd: 3 - Pass type: 0x0, config: 0x1, fd: -1 - Pass type: 0x4, config: 0x76, fd: 3 - Pass type: 0x4, config: 0xc1, fd: 3 - Pass type: 0x4, config: 0x12, fd: -1 - Pass test child finished with 0 ---- end ---- AMD IBS via core pmu: Ok
--- tools/perf/tests/Build | 1 + tools/perf/tests/amd-ibs-via-core-pmu.c | 72 +++++++++++++++++++++++++ tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/tests.h | 1 + 4 files changed, 75 insertions(+) create mode 100644 tools/perf/tests/amd-ibs-via-core-pmu.c
tools/perf/tests/Build +1 -0
@@ -69,6 +69,7 @@ perf-y += dlfilter-test.o perf-y += sigtrap.o perf-y += event_groups.o perf-y += symbols.o +perf-y += amd-ibs-via-core-pmu.o $(OUTPUT)tests/llvm-src-base.c: tests/bpf-script-example.c tests/Build $(call rule_mkdir)
tools/perf/tests/amd-ibs-via-core-pmu.c new file +72 -0
72 lines not on this page · raw message
tools/perf/tests/builtin-test.c +1 -0
@@ -119,6 +119,7 @@ static struct test_suite *generic_tests[] = { &suite__sigtrap, &suite__event_groups, &suite__symbols, + &suite__amd_ibs_via_core_pmu, NULL, };
tools/perf/tests/tests.h +1 -0
@@ -149,6 +149,7 @@ DECLARE_SUITE(dlfilter);DECLARE_SUITE (sigtrap);DECLARE_SUITE (event_groups);DECLARE_SUITE (symbols); +DECLARE_SUITE (amd_ibs_via_core_pmu);/* * PowerPC and S390 do not support creation of instruction breakpoints using the
Signature · 1 line
2.40.0
Ian Rogers
reviewerRe: [PATCH v3 3/3] perf test: Add selftest to test IBS invocation via core pmu events
Quoted 34 lines · Ravi Bangoria
IBS pmu can be invoked via fixed set of core pmu events with '
precise_ip' set to 1. Add a simple event open test for all these events.Without kernel fix:
$ sudo ./perf test -vv 76 76: AMD IBS via core pmu : --- start --- test child forked, pid 6553 Using CPUID AuthenticAMD-25-1-1 type: 0x0, config: 0x0, fd: 3 - Pass type: 0x0, config: 0x1, fd: -1 - Pass type: 0x4, config: 0x76, fd: -1 - Fail type: 0x4, config: 0xc1, fd: -1 - Fail type: 0x4, config: 0x12, fd: -1 - Pass test child finished with -1 ---- end ---- AMD IBS via core pmu: FAILED!With kernel fix:
$ sudo ./perf test -vv 76 76: AMD IBS via core pmu : --- start --- test child forked, pid 7526 Using CPUID AuthenticAMD-25-1-1 type: 0x0, config: 0x0, fd: 3 - Pass type: 0x0, config: 0x1, fd: -1 - Pass type: 0x4, config: 0x76, fd: 3 - Pass type: 0x4, config: 0xc1, fd: 3 - Pass type: 0x4, config: 0x12, fd: -1 - Pass test child finished with 0 ---- end ---- AMD IBS via core pmu: Ok
Quoted 1 line
Signed-off-by Ravi Bangoria <[email protected]>
Thanks Ravi, as the test is AMD specific I think it makes sense to place it in:tools/perf/arch/x86/tests
and then to update the test list in:tools/perf/arch/x86/tests/arch-tests.c
Thanks,
Ian
Quoted 125 lines
--- tools/perf/tests/Build | 1 + tools/perf/tests/amd-ibs-via-core-pmu.c | 72 +++++++++++++++++++++++++ tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/tests.h | 1 + 4 files changed, 75 insertions(+) create mode 100644 tools/perf/tests/amd-ibs-via-core-pmu.ctools/perf/tests/Build +1
-0@@ -69,6 +69,7 @@ perf-y += dlfilter-test.o perf-y += sigtrap.o perf-y += event_groups.o perf-y += symbols.o +perf-y += amd-ibs-via-core-pmu.o $(OUTPUT)tests/llvm-src-base.c: tests/bpf-script-example.c tests/Build $(call rule_mkdir)tools/perf/tests/amd-ibs-via-core-pmu.c new file +72
-0@@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "linux/perf_event.h" +#include "tests.h" +#include "pmu.h" +#include "pmus.h" +#include "../perf-sys.h" +#include "debug.h" + +#define NR_SUB_TESTS 5 + +static struct sub_tests { +int type; +unsigned long config; +bool valid; +} sub_tests[NR_SUB_TESTS ] = { + {PERF_TYPE_HARDWARE ,PERF_COUNT_HW_CPU_CYCLES , true }, + {PERF_TYPE_HARDWARE ,PERF_COUNT_HW_INSTRUCTIONS , false }, + {PERF_TYPE_RAW ,0x076 , true }, + {PERF_TYPE_RAW ,0x0C1 , true }, + {PERF_TYPE_RAW ,0x012 , false }, +}; + +static int event_open (int type,unsigned long config) +{ +struct perf_event_attr attr; + +memset (&attr,0 ,sizeof (struct perf_event_attr )); + attr.type = type; + attr.size =sizeof (struct perf_event_attr ); + attr.config = config; + attr.disabled =1 ; + attr.precise_ip =1 ; + attr.sample_type =PERF_SAMPLE_IP |PERF_SAMPLE_TID ; + attr.sample_period =100000 ; + +return sys_perf_event_open (&attr,-1 ,0 ,-1 ,0 ); +} + +static int test__amd_ibs_via_core_pmu (struct test_suite *text__maybe_unused , +int subtest__maybe_unused ) +{ +struct perf_pmu *ibs_pmu; +int ret =TEST_OK ; +int fd, i; + +if (list_empty (&pmus)) +perf_pmu__scan (NULL ); + + ibs_pmu =perf_pmu__find ("ibs_op" ); +if (!ibs_pmu) +return TEST_SKIP ; + +for (i =0 ; i <NR_SUB_TESTS ; i++) { + fd =event_open (sub_tests[i].type, sub_tests[i].config); +pr_debug ("type: 0x%x, config: 0x%lx, fd: %d - " , sub_tests[i].type, + sub_tests[i].config, fd); +if ((sub_tests[i].valid && fd ==-1 ) || + (!sub_tests[i].valid && fd >0 )) { +pr_debug ("Fail\n" ); + ret =TEST_FAIL ; + }else { +pr_debug ("Pass\n" ); + } + +if (fd >0 ) +close (fd); + } + +return ret; +} + +DEFINE_SUITE ("AMD IBS via core pmu" , amd_ibs_via_core_pmu);tools/perf/tests/builtin-test.c +1
-0@@ -119,6 +119,7 @@ static struct test_suite *generic_tests[] = { &suite__sigtrap, &suite__event_groups, &suite__symbols, + &suite__amd_ibs_via_core_pmu, NULL, };tools/perf/tests/tests.h +1
-1@@ -149,6 +149,7 @@ DECLARE_SUITE(dlfilter);DECLARE_SUITE (sigtrap);DECLARE_SUITE (event_groups);DECLARE_SUITE (symbols); +DECLARE_SUITE (amd_ibs_via_core_pmu);/* * PowerPC and S390 do not support creation of instruction breakpoints using the -- 2.40.0
Re: [PATCH v3 3/3] perf test: Add selftest to test IBS invocation via core pmu events
Quoted 35 lines · Ian Rogers
Quoted 34 lines · Ravi Bangoria
IBS pmu can be invoked via fixed set of core pmu events with '
precise_ip' set to 1. Add a simple event open test for all these events.Without kernel fix:
$ sudo ./perf test -vv 76 76: AMD IBS via core pmu : --- start --- test child forked, pid 6553 Using CPUID AuthenticAMD-25-1-1 type: 0x0, config: 0x0, fd: 3 - Pass type: 0x0, config: 0x1, fd: -1 - Pass type: 0x4, config: 0x76, fd: -1 - Fail type: 0x4, config: 0xc1, fd: -1 - Fail type: 0x4, config: 0x12, fd: -1 - Pass test child finished with -1 ---- end ---- AMD IBS via core pmu: FAILED!With kernel fix:
$ sudo ./perf test -vv 76 76: AMD IBS via core pmu : --- start --- test child forked, pid 7526 Using CPUID AuthenticAMD-25-1-1 type: 0x0, config: 0x0, fd: 3 - Pass type: 0x0, config: 0x1, fd: -1 - Pass type: 0x4, config: 0x76, fd: 3 - Pass type: 0x4, config: 0xc1, fd: 3 - Pass type: 0x4, config: 0x12, fd: -1 - Pass test child finished with 0 ---- end ---- AMD IBS via core pmu: Ok
Quoted 6 lines
Quoted 1 line
Signed-off-by Ravi Bangoria <[email protected]>Thanks Ravi, as the test is AMD specific I think it makes sense to place it in:
tools/perf/arch/x86/tests
and then to update the test list in:tools/perf/arch/x86/tests/arch-tests.c
Sure, will respin.
Thanks,
Ravi