perf tests: Fix tests in 'Parse event definition strings'
PATCH 1/1 perf tests: Fix tests in 'Parse event definition strings'
From: Tinghao Zhang <[email protected]>
The 'Test event parsing' test set under 'Parse event definion strings' does not apply to hybrid events. Since hybrid events have a dedicated test set 'Test parsing of hybrid CPU events', skip these tests on hybrid platforms.
--- tools/perf/tests/parse-events.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tools/perf/tests/parse-events.c +4 -1
@@ -2146,6 +2146,9 @@ static int test_events(const struct evlist_test *events, int cnt)static int test__events2 (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) { +if (perf_pmu__has_hybrid ()) +return TEST_SKIP ; +return test_events (test__events,ARRAY_SIZE (test__events)); }
@@ -2421,7 +2424,7 @@ static int test__pmu_events_alias2(struct test_suite *test __maybe_unused,static struct test_case tests__parse_events[] = {TEST_CASE_REASON ("Test event parsing" , events2, -"permissions" ), +"permissions or hybrid" ),TEST_CASE_REASON ("Test parsing of \"hybrid\" CPU events" , hybrid,"not hybrid" ),
Signature · 1 line
2.34.1
Quoted 10 lines · On 2023-04-11 5:43 a.m., [email protected] wrote:
From: Tinghao Zhang <[email protected]>
The 'Test event parsing' test set under 'Parse event definion strings' does not apply to hybrid events. Since hybrid events have a dedicated test set 'Test parsing of hybrid CPU events', skip these tests on hybrid platforms.
Fixes 7741e03e808a ("perf test: Parse events break apart tests")Reported-by Yi Ammy <[email protected]>Signed-off-by Tinghao Zhang <[email protected]>
This patch looks good to me.
Thanks,
Kan
Quoted 27 lines
--- tools/perf/tests/parse-events.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)tools/perf/tests/parse-events.c +4
-1@@ -2146,6 +2146,9 @@ static int test_events(const struct evlist_test *events, int cnt)static int test__events2 (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) { +if (perf_pmu__has_hybrid ()) +return TEST_SKIP ; +return test_events (test__events,ARRAY_SIZE (test__events)); }@@ -2421,7 +2424,7 @@ static int test__pmu_events_alias2(struct test_suite *test __maybe_unused,static struct test_case tests__parse_events[] = {TEST_CASE_REASON ("Test event parsing" , events2, -"permissions" ), +"permissions or hybrid" ),TEST_CASE_REASON ("Test parsing of \"hybrid\" CPU events" , hybrid,"not hybrid" ),
A message that is not in the archive (sent off-list or not yet received): CAP-5=fVWx7F9D0MvU=tCJ4nQ3QjaB7fVaCjvD8j4F-1yo5piDA@mail.gmail.com
A message that is not in the archive (sent off-list or not yet received): [email protected]
Ian Rogers
reviewerQuoted 20 lines · Tinghao
On 4/11/2023 9:50 PM, Ian Rogers wrote:
Quoted 11 lines · On Tue, Apr 11, 2023, 2:49 AM <[email protected]> wrote:
From: Tinghao Zhang <[email protected]>
The 'Test event parsing' test set under 'Parse event definion strings' does not apply to hybrid events. Since hybrid events have a dedicated test set 'Test parsing of hybrid CPU events', skip these tests on hybrid platforms.
Fixes 7741e03e808a ("perf test: Parse events break apart tests")Reported-by Yi Ammy <[email protected]>Signed-off-by Tinghao Zhang <[email protected]>This feels wrong and could make a different problem. Could you send the failure you are seeing? There is a verbose flag (-vv) that you can pass to perf test.
Quoted 7 lines
Thanks,
IanHi Ian,
I think these test cases are not designed for hybrid platforms. As the current implementation, if one general event type is used on hybrid platform, perf tool would generate seperate event for each pmu. But tests in 'Test event parsing' asserts only one evlist entry per event type. Another problem is that for
Hi Tinghao,
So you are right that the tests weren't designed for hybrid platforms because they didn't exist when the tests were created, but the bigger issue is what should perf be doing in these cases. On ARM's BIG.little version of "hybrid" there is a single PMU and generic events that work on both kind of core. But even with a single core we can get into issues that an event can be defined for more than one PMU. For example, a GPU may have an instructions-retired count. In sysfs we'd have /sys/devices/my-gpu and /sys/devices/cpu. If the user asks for instructions-retired and has suitable permission, should the perf tool display instructions-retired for the GPU or just the cpu? My belief is that we should display my-gpu/instructions-retired/ and cpu/instructions-retired/. If the user wants to then limit to the CPU one, they can use the full PMU and event name cpu/instructions-retired/.
Why does this matter? Well if we have a common plan for how to deal with events that match on more than 1 PMU then hybrid can just follow the plan and we can remove all of the hybrid code from the perf tool. Yes, the tests made an assumption there'd only be 1 of a kind of PMU for an event, so we need to change the test to assume that there will be at least 1, or that the number should match the number of PMUs or core PMUs.
Quoted 3 lines
PERF_TYPE_HARDWAREandPERF_TYPE_HW_CACHE, attr.config has a different format for hybrid events, so we see attr.config errors. Also, it is not applicable to put events from different PMUs to one group, but here some tests put general events in one group. As for the patch that first introduces hybrid test cases2541cb63ac0c("perf tests: Add hybrid cases for 'Parse event definition strings' test") , this test set is skipped. I think it's better if we skip this test set for hybrid pmus.
So I think:2541cb63ac0c ("perf tests: Add hybrid cases for 'Parse event definition strings' test")
was added quickly in order that nothing be blocked, but with hindsight it regrettably didn't make a generic solution that could be used for a range of PMUs. You may be interested to see that the problem of being too CPU centric for perf is something reported by Intel's GPU team:
https://www.kernel.org/doc/html/next/gpu/i915.html#issues-hit-with-first-prototype-based-on-core-perf
I'd rather that the code were not hybrid or CPU specific, but everything was based off the PMU abstraction. For the attributes we'll need to improve the test to make it more generic to handle matching on multiple PMUs.
Quoted 145 lines
Below is the perf test result on one Intel Meteor Lake platform
6: Parse event definition strings :
6.1: Test event parsing :
--- start ---
test child forked, pid 44965
running test 0 'syscalls:sys_enter_openat'
Using CPUID GenuineIntel-6-AA-2intel_ptdefault config: tsc,mtc,mtc_period=3,psb_period=3,pt,branch running test 1 'syscalls:*'
running test 2 'r1a'
FAILEDtests/parse-events.c:87wrong number of entries
Event test failure: test 2 'r1a'running test 3 '1:1'
running test 4 'instructions'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 4 'instructions'running test 5 'cycles/period=100000,config2/' FAILEDtests/parse-events.c:118wrong number of entries
Event test failure: test 5 'cycles/period=100000,config2/'running test 6 'faults' running test 7 'L1-dcache-load-miss'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 3
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
FAILEDtests/parse-events.c:152wrong config
Event test failure: test 7 'L1-dcache-load-miss'running test 8 'mem:0' running test 9 'mem:0:x'
running test 10 'mem:0:r'
running test 11 'mem:0:w'
running test 12 'syscalls:sys_enter_openat:k'
running test 13 'syscalls:*:u'
running test 14 'r1a:kp'
FAILEDtests/parse-events.c:87wrong number of entries Event test failure: test 14 'r1a:kp'running test 15 '1:1:hp' running test 16 'instructions:h'
FAILEDtests/parse-events.c:107wrong number of entries Event test failure: test 16 'instructions:h'running test 17 'faults:u' running test 18 'L1-dcache-load-miss:kp'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 39
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
FAILEDtests/parse-events.c:152wrong config
Event test failure: test 18 'L1-dcache-load-miss:kp'running test 19 'mem:0:u'
running test 20 'mem:0:x:k'
running test 21 'mem:0:r:hp'
running test 22 'mem:0:w:up'
running test 23 'r1,syscalls:sys_enter_openat:k,1:1:hp'
FAILEDtests/parse-events.c:466wrong number of entries
Event test failure: test 23 'r1,syscalls:sys_enter_openat:k,1:1:hp'running test 24 'instructions:G'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 24 'instructions:G'running test 25 'instructions:H'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 25 'instructions:H'running test 26 'mem:0:rw'
running test 27 'mem:0:rw:kp'
running test 28 '{instructions:k,cycles:upp}'
FAILEDtests/parse-events.c:679wrong number of entries
Event test failure: test 28 '{instructions:k,cycles:upp}'running test 29 '{faults:k,cache-references}:u,cycles:k'
FAILEDtests/parse-events.c:721wrong number of entries
Event test failure: test 29 '{faults:k,cache-references}:u,cycles:k'running test 30 'group1{syscalls:sys_enter_openat:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u' FAILEDtests/parse-events.c:777wrong number of entries
Event test failure: test 30 'group1{syscalls:sys_enter_openat:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u'running test 31 '{cycles:u,instructions:kp}:p' FAILEDtests/parse-events.c:870wrong number of entries
Event test failure: test 31 '{cycles:u,instructions:kp}:p'running test 32 '{cycles,instructions}:G,{cycles:G,instructions:G},cycles'
FAILEDtests/parse-events.c:914wrong number of entries
Event test failure: test 32 '{cycles,instructions}:G,{cycles:G,instructions:G},cycles'running test 33 '*:*'
running test 34 '{cycles,cache-misses:G}:H'
FAILEDtests/parse-events.c:1000wrong number of entries
Event test failure: test 34 '{cycles,cache-misses:G}:H'running test 35 '{cycles,cache-misses:H}:G'
FAILEDtests/parse-events.c:1040wrong number of entries
Event test failure: test 35 '{cycles,cache-misses:H}:G'running test 36 '{cycles:G,cache-misses:H}:u'
FAILEDtests/parse-events.c:1080wrong number of entries
Event test failure: test 36 '{cycles:G,cache-misses:H}:u'running test 37 '{cycles:G,cache-misses:H}:uG'
FAILEDtests/parse-events.c:1120wrong number of entries
Event test failure: test 37 '{cycles:G,cache-misses:H}:uG'running test 38 '{cycles,cache-misses,branch-misses}:S'
FAILEDtests/parse-events.c:1160wrong number of entries
Event test failure: test 38 '{cycles,cache-misses,branch-misses}:S'running test 39 '{instructions,branch-misses}:Su'
FAILEDtests/parse-events.c:1213wrong number of entries
Event test failure: test 39 '{instructions,branch-misses}:Su'running test 40 'instructions:uDp'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 40 'instructions:uDp'running test 41 '{cycles,cache-misses,branch-misses}:D'
FAILEDtests/parse-events.c:1265wrong number of entries
Event test failure: test 41 '{cycles,cache-misses,branch-misses}:D'running test 42 'mem:0/1'
running test 43 'mem:0/2:w'
running test 44 'mem:0/4:rw:u'
running test 45 'instructions:I'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 45 'instructions:I'running test 46 'instructions:kIG'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 46 'instructions:kIG'running test 47 'task-clock:P,cycles'
FAILEDtests/parse-events.c:1382wrong number of entries
Event test failure: test 47 'task-clock:P,cycles'running test 48 'instructions/name=insn/'
running test 49 'r1234/name=rawpmu/'
running test 50 '4:0x6530160/name=numpmu/'
WARNING: event 'numpmu' not valid (bits 16-17,20,22 of config '6530160' not supported by kernel)!
running test 51 'L1-dcache-misses/name=cachepmu/'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 40
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
running test 52 'intel_pt//u'
running test 53 'cycles/name='COMPLEX_CYCLES_NAME:orig=cycles,desc=chip-clock-ticks'/Duk' running test 54 'cycles//u'
FAILEDtests/parse-events.c:1457wrong config
Event test failure: test 54 'cycles//u'running test 55 'cycles:k'
FAILEDtests/parse-events.c:1467wrong config
Event test failure: test 55 'cycles:k'running test 56 'instructions:uep'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 56 'instructions:uep'running test 57 '{cycles,cache-misses,branch-misses}:e' FAILEDtests/parse-events.c:1309wrong number of entries
Event test failure: test 57 '{cycles,cache-misses,branch-misses}:e'test child finished with -1 ---- end ----
Quoted 1 line
Parse event definition strings subtest 1: FAILED!
Thanks! So "wrong number of entries" is the dominant failure and should be straightforward to make more PMU agnostic. Tests that are tied to a PMU like:
running test 52 'intel_pt//u'
succeed and it seems sad to lose coverage of them with this change. Overall, I think we should work to remove the hybrid test and make sure we get coverage in the general test. That means just skipping the test is the wrong thing to do.
Thanks,
Ian
Quoted 34 lines
Thanks,
Tinghao
Quoted 31 lines
--- tools/perf/tests/parse-events.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)tools/perf/tests/parse-events.c +4
-2@@ -2146,6 +2146,9 @@ static int test_events(const struct evlist_test *events, int cnt)static int test__events2 (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) { +if (perf_pmu__has_hybrid ()) +return TEST_SKIP ; +return test_events (test__events,ARRAY_SIZE (test__events)); }@@ -2421,7 +2424,7 @@ static int test__pmu_events_alias2(struct test_suite *test __maybe_unused,static struct test_case tests__parse_events[] = {TEST_CASE_REASON ("Test event parsing" , events2, -"permissions" ), +"permissions or hybrid" ),TEST_CASE_REASON ("Test parsing of \"hybrid\" CPU events" , hybrid,"not hybrid" ), --2.34.1
Quoted 30 lines · Ian Rogers
Quoted 20 lines · Tinghao
On 4/11/2023 9:50 PM, Ian Rogers wrote:
Quoted 11 lines · On Tue, Apr 11, 2023, 2:49 AM <[email protected]> wrote:
From: Tinghao Zhang <[email protected]>
The 'Test event parsing' test set under 'Parse event definion strings' does not apply to hybrid events. Since hybrid events have a dedicated test set 'Test parsing of hybrid CPU events', skip these tests on hybrid platforms.
Fixes 7741e03e808a ("perf test: Parse events break apart tests")Reported-by Yi Ammy <[email protected]>Signed-off-by Tinghao Zhang <[email protected]>This feels wrong and could make a different problem. Could you send the failure you are seeing? There is a verbose flag (-vv) that you can pass to perf test.
Quoted 7 lines
Thanks,
IanHi Ian,
I think these test cases are not designed for hybrid platforms. As the current implementation, if one general event type is used on hybrid platform, perf tool would generate seperate event for each pmu. But tests in 'Test event parsing' asserts only one evlist entry per event type. Another problem is that for
Quoted 7 lines
Hi Tinghao,
So you are right that the tests weren't designed for hybrid platforms because they didn't exist when the tests were created, but the bigger issue is what should perf be doing in these cases. On ARM's BIG.little version of "hybrid" there is a single PMU and generic events that work on both kind of core.
But we have different PMUs for Intel hybrid. The event list is quite different between e-core and p-core. There are some features which only available for one type of core. I don't think we can have a single PMU for Intel hybrid.
Quoted 10 lines
But even with a single core we can get into
issues that an event can be defined for more than one PMU. For example, a GPU may have an instructions-retired count. In sysfs we'd have/sys/devices/my-gpuand/sys/devices/cpu. If the user asks for instructions-retired and has suitable permission, should the perf tool display instructions-retired for the GPU or just the cpu? My belief is that we should display my-gpu/instructions-retired/ and cpu/instructions-retired/. If the user wants to then limit to the CPU one, they can use the full PMU and event name cpu/instructions-retired/.
Quoted 7 lines
Quoted 1 line
Why does this matter? Well if we have a common plan for how to deal
with events that match on more than 1 PMU then hybrid can just follow the plan and we can remove all of the hybrid code from the perf tool. Yes, the tests made an assumption there'd only be 1 of a kind of PMU for an event, so we need to change the test to assume that there will be at least 1, or that the number should match the number of PMUs or core PMUs.
How about the group? Using weak group?
Quoted 5 lines
Quoted 4 lines
PERF_TYPE_HARDWAREandPERF_TYPE_HW_CACHE, attr.config has a different format for hybrid events, so we see attr.config errors. Also, it is not applicable to put events from different PMUs to one group, but here some tests put general events in one group. As for the patch that first introduces hybrid test cases2541cb63ac0c("perf tests: Add hybrid cases for 'Parse event definition strings' test") , this test set is skipped. I think it's better if we skip this test set for hybrid pmus.
Quoted 12 lines
So I think:
2541cb63ac0c("perf tests: Add hybrid cases for 'Parse event definition strings' test")
was added quickly in order that nothing be blocked, but with hindsight it regrettably didn't make a generic solution that could be used for a range of PMUs. You may be interested to see that the problem of being too CPU centric for perf is something reported by Intel's GPU team:
https://www.kernel.org/doc/html/next/gpu/i915.html#issues-hit-with-first-prototype-based-on-core-perf
I'd rather that the code were not hybrid or CPU specific, but everything was based off the PMU abstraction. For the attributes we'll need to improve the test to make it more generic to handle matching on multiple PMUs.
The current "test__events" doesn't generic enough. There is L1-dcache-load-miss which apparently CPU specific. Also ARCH specific events, e.g., kvm-s390, intel_pt. And groups.
How should we make it generic? Any suggestions?
Thanks,
Kan
Quoted 196 lines
Quoted 146 lines
Below is the perf test result on one Intel Meteor Lake platform
6: Parse event definition strings :
6.1: Test event parsing :
--- start ---
test child forked, pid 44965
running test 0 'syscalls:sys_enter_openat'
Using CPUID GenuineIntel-6-AA-2intel_ptdefault config: tsc,mtc,mtc_period=3,psb_period=3,pt,branch running test 1 'syscalls:*'
running test 2 'r1a'
FAILEDtests/parse-events.c:87wrong number of entries
Event test failure: test 2 'r1a'running test 3 '1:1'
running test 4 'instructions'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 4 'instructions'running test 5 'cycles/period=100000,config2/' FAILEDtests/parse-events.c:118wrong number of entries
Event test failure: test 5 'cycles/period=100000,config2/'running test 6 'faults' running test 7 'L1-dcache-load-miss'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 3
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
FAILEDtests/parse-events.c:152wrong config
Event test failure: test 7 'L1-dcache-load-miss'running test 8 'mem:0' running test 9 'mem:0:x'
running test 10 'mem:0:r'
running test 11 'mem:0:w'
running test 12 'syscalls:sys_enter_openat:k'
running test 13 'syscalls:*:u'
running test 14 'r1a:kp'
FAILEDtests/parse-events.c:87wrong number of entries Event test failure: test 14 'r1a:kp'running test 15 '1:1:hp' running test 16 'instructions:h'
FAILEDtests/parse-events.c:107wrong number of entries Event test failure: test 16 'instructions:h'running test 17 'faults:u' running test 18 'L1-dcache-load-miss:kp'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 39
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
FAILEDtests/parse-events.c:152wrong config
Event test failure: test 18 'L1-dcache-load-miss:kp'running test 19 'mem:0:u'
running test 20 'mem:0:x:k'
running test 21 'mem:0:r:hp'
running test 22 'mem:0:w:up'
running test 23 'r1,syscalls:sys_enter_openat:k,1:1:hp'
FAILEDtests/parse-events.c:466wrong number of entries
Event test failure: test 23 'r1,syscalls:sys_enter_openat:k,1:1:hp'running test 24 'instructions:G'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 24 'instructions:G'running test 25 'instructions:H'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 25 'instructions:H'running test 26 'mem:0:rw'
running test 27 'mem:0:rw:kp'
running test 28 '{instructions:k,cycles:upp}'
FAILEDtests/parse-events.c:679wrong number of entries
Event test failure: test 28 '{instructions:k,cycles:upp}'running test 29 '{faults:k,cache-references}:u,cycles:k'
FAILEDtests/parse-events.c:721wrong number of entries
Event test failure: test 29 '{faults:k,cache-references}:u,cycles:k'running test 30 'group1{syscalls:sys_enter_openat:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u' FAILEDtests/parse-events.c:777wrong number of entries
Event test failure: test 30 'group1{syscalls:sys_enter_openat:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u'running test 31 '{cycles:u,instructions:kp}:p' FAILEDtests/parse-events.c:870wrong number of entries
Event test failure: test 31 '{cycles:u,instructions:kp}:p'running test 32 '{cycles,instructions}:G,{cycles:G,instructions:G},cycles'
FAILEDtests/parse-events.c:914wrong number of entries
Event test failure: test 32 '{cycles,instructions}:G,{cycles:G,instructions:G},cycles'running test 33 '*:*'
running test 34 '{cycles,cache-misses:G}:H'
FAILEDtests/parse-events.c:1000wrong number of entries
Event test failure: test 34 '{cycles,cache-misses:G}:H'running test 35 '{cycles,cache-misses:H}:G'
FAILEDtests/parse-events.c:1040wrong number of entries
Event test failure: test 35 '{cycles,cache-misses:H}:G'running test 36 '{cycles:G,cache-misses:H}:u'
FAILEDtests/parse-events.c:1080wrong number of entries
Event test failure: test 36 '{cycles:G,cache-misses:H}:u'running test 37 '{cycles:G,cache-misses:H}:uG'
FAILEDtests/parse-events.c:1120wrong number of entries
Event test failure: test 37 '{cycles:G,cache-misses:H}:uG'running test 38 '{cycles,cache-misses,branch-misses}:S'
FAILEDtests/parse-events.c:1160wrong number of entries
Event test failure: test 38 '{cycles,cache-misses,branch-misses}:S'running test 39 '{instructions,branch-misses}:Su'
FAILEDtests/parse-events.c:1213wrong number of entries
Event test failure: test 39 '{instructions,branch-misses}:Su'running test 40 'instructions:uDp'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 40 'instructions:uDp'running test 41 '{cycles,cache-misses,branch-misses}:D'
FAILEDtests/parse-events.c:1265wrong number of entries
Event test failure: test 41 '{cycles,cache-misses,branch-misses}:D'running test 42 'mem:0/1'
running test 43 'mem:0/2:w'
running test 44 'mem:0/4:rw:u'
running test 45 'instructions:I'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 45 'instructions:I'running test 46 'instructions:kIG'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 46 'instructions:kIG'running test 47 'task-clock:P,cycles'
FAILEDtests/parse-events.c:1382wrong number of entries
Event test failure: test 47 'task-clock:P,cycles'running test 48 'instructions/name=insn/'
running test 49 'r1234/name=rawpmu/'
running test 50 '4:0x6530160/name=numpmu/'
WARNING: event 'numpmu' not valid (bits 16-17,20,22 of config '6530160' not supported by kernel)!
running test 51 'L1-dcache-misses/name=cachepmu/'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 40
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
running test 52 'intel_pt//u'
running test 53 'cycles/name='COMPLEX_CYCLES_NAME:orig=cycles,desc=chip-clock-ticks'/Duk' running test 54 'cycles//u'
FAILEDtests/parse-events.c:1457wrong config
Event test failure: test 54 'cycles//u'running test 55 'cycles:k'
FAILEDtests/parse-events.c:1467wrong config
Event test failure: test 55 'cycles:k'running test 56 'instructions:uep'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 56 'instructions:uep'running test 57 '{cycles,cache-misses,branch-misses}:e' FAILEDtests/parse-events.c:1309wrong number of entries
Event test failure: test 57 '{cycles,cache-misses,branch-misses}:e'test child finished with -1 ---- end ----Quoted 1 line
Parse event definition strings subtest 1: FAILED!
Thanks! So "wrong number of entries" is the dominant failure and should be straightforward to make more PMU agnostic. Tests that are tied to a PMU like:
running test 52 'intel_pt//u'
succeed and it seems sad to lose coverage of them with this change. Overall, I think we should work to remove the hybrid test and make sure we get coverage in the general test. That means just skipping the test is the wrong thing to do.
Thanks,
IanQuoted 34 lines
Thanks,
Tinghao
Quoted 31 lines
--- tools/perf/tests/parse-events.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)tools/perf/tests/parse-events.c +4
-2@@ -2146,6 +2146,9 @@ static int test_events(const struct evlist_test *events, int cnt)static int test__events2 (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) { +if (perf_pmu__has_hybrid ()) +return TEST_SKIP ; +return test_events (test__events,ARRAY_SIZE (test__events)); }@@ -2421,7 +2424,7 @@ static int test__pmu_events_alias2(struct test_suite *test __maybe_unused,static struct test_case tests__parse_events[] = {TEST_CASE_REASON ("Test event parsing" , events2, -"permissions" ), +"permissions or hybrid" ),TEST_CASE_REASON ("Test parsing of \"hybrid\" CPU events" , hybrid,"not hybrid" ), --2.34.1
Ian Rogers
reviewerQuoted 42 lines · Kan
Quoted 30 lines · Ian Rogers
Quoted 20 lines · Tinghao
On 4/11/2023 9:50 PM, Ian Rogers wrote:
Quoted 11 lines · On Tue, Apr 11, 2023, 2:49 AM <[email protected]> wrote:
From: Tinghao Zhang <[email protected]>
The 'Test event parsing' test set under 'Parse event definion strings' does not apply to hybrid events. Since hybrid events have a dedicated test set 'Test parsing of hybrid CPU events', skip these tests on hybrid platforms.
Fixes 7741e03e808a ("perf test: Parse events break apart tests")Reported-by Yi Ammy <[email protected]>Signed-off-by Tinghao Zhang <[email protected]>This feels wrong and could make a different problem. Could you send the failure you are seeing? There is a verbose flag (-vv) that you can pass to perf test.
Quoted 7 lines
Thanks,
IanHi Ian,
I think these test cases are not designed for hybrid platforms. As the current implementation, if one general event type is used on hybrid platform, perf tool would generate seperate event for each pmu. But tests in 'Test event parsing' asserts only one evlist entry per event type. Another problem is that for
Quoted 7 lines
Hi Tinghao,
So you are right that the tests weren't designed for hybrid platforms because they didn't exist when the tests were created, but the bigger issue is what should perf be doing in these cases. On ARM's BIG.little version of "hybrid" there is a single PMU and generic events that work on both kind of core.
Quoted 4 lines
But we have different PMUs for Intel hybrid. The event list is quite different between e-core and p-core. There are some features which only available for one type of core. I don't think we can have a single PMU for Intel hybrid.
So we make the test and the expectation match.
Quoted 11 lines
Quoted 11 lines
But even with a single core we can get into
issues that an event can be defined for more than one PMU. For example, a GPU may have an instructions-retired count. In sysfs we'd have/sys/devices/my-gpuand/sys/devices/cpu. If the user asks for instructions-retired and has suitable permission, should the perf tool display instructions-retired for the GPU or just the cpu? My belief is that we should display my-gpu/instructions-retired/ and cpu/instructions-retired/. If the user wants to then limit to the CPU one, they can use the full PMU and event name cpu/instructions-retired/.Quoted 1 line
Why does this matter? Well if we have a common plan for how to deal
Quoted 8 lines
Quoted 6 lines
with events that match on more than 1 PMU then hybrid can just follow the plan and we can remove all of the hybrid code from the perf tool. Yes, the tests made an assumption there'd only be 1 of a kind of PMU for an event, so we need to change the test to assume that there will be at least 1, or that the number should match the number of PMUs or core PMUs.
How about the group? Using weak group?
No need, just make sure to call parse_events__sort_events_and_fix_groups:
https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/util/parse-events.c?h=perf-tools-next#n2167
Quoted 18 lines
Quoted 5 lines
Quoted 4 lines
PERF_TYPE_HARDWAREandPERF_TYPE_HW_CACHE, attr.config has a different format for hybrid events, so we see attr.config errors. Also, it is not applicable to put events from different PMUs to one group, but here some tests put general events in one group. As for the patch that first introduces hybrid test cases2541cb63ac0c("perf tests: Add hybrid cases for 'Parse event definition strings' test") , this test set is skipped. I think it's better if we skip this test set for hybrid pmus.Quoted 12 lines
So I think:
2541cb63ac0c("perf tests: Add hybrid cases for 'Parse event definition strings' test")
was added quickly in order that nothing be blocked, but with hindsight it regrettably didn't make a generic solution that could be used for a range of PMUs. You may be interested to see that the problem of being too CPU centric for perf is something reported by Intel's GPU team:
https://www.kernel.org/doc/html/next/gpu/i915.html#issues-hit-with-first-prototype-based-on-core-perf
I'd rather that the code were not hybrid or CPU specific, but everything was based off the PMU abstraction. For the attributes we'll need to improve the test to make it more generic to handle matching on multiple PMUs.
Quoted 5 lines
The current "
test__events" doesn't generic enough. There is L1-dcache-load-miss which apparently CPU specific. Also ARCH specific events, e.g., kvm-s390,intel_pt. And groups.How should we make it generic? Any suggestions?
s390 uses #ifdefs, intel_pt uses the intel_pt_valid call to enable the test - this returns false if the intel_pt PMU isn't detected. I have a fix for legacy events like L1-dcache-load-miss, the hybrid hack of having the parser run the parser inside itself is gross, we should just fix the parser. But given use of these legacy events is marginal/deprecated it isn't clear we need to support them on hybrid. Fwiw, I want to refactor the parsing code into a generic event to perf_event_attr library that is more permissible than GPLv2, so I need to find some time to make it happen.
Thanks,
Ian
Quoted 199 lines
Thanks,
KanQuoted 196 lines
Quoted 146 lines
Below is the perf test result on one Intel Meteor Lake platform
6: Parse event definition strings :
6.1: Test event parsing :
--- start ---
test child forked, pid 44965
running test 0 'syscalls:sys_enter_openat'
Using CPUID GenuineIntel-6-AA-2intel_ptdefault config: tsc,mtc,mtc_period=3,psb_period=3,pt,branch running test 1 'syscalls:*'
running test 2 'r1a'
FAILEDtests/parse-events.c:87wrong number of entries
Event test failure: test 2 'r1a'running test 3 '1:1'
running test 4 'instructions'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 4 'instructions'running test 5 'cycles/period=100000,config2/' FAILEDtests/parse-events.c:118wrong number of entries
Event test failure: test 5 'cycles/period=100000,config2/'running test 6 'faults' running test 7 'L1-dcache-load-miss'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 3
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
FAILEDtests/parse-events.c:152wrong config
Event test failure: test 7 'L1-dcache-load-miss'running test 8 'mem:0' running test 9 'mem:0:x'
running test 10 'mem:0:r'
running test 11 'mem:0:w'
running test 12 'syscalls:sys_enter_openat:k'
running test 13 'syscalls:*:u'
running test 14 'r1a:kp'
FAILEDtests/parse-events.c:87wrong number of entries Event test failure: test 14 'r1a:kp'running test 15 '1:1:hp' running test 16 'instructions:h'
FAILEDtests/parse-events.c:107wrong number of entries Event test failure: test 16 'instructions:h'running test 17 'faults:u' running test 18 'L1-dcache-load-miss:kp'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 39
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
FAILEDtests/parse-events.c:152wrong config
Event test failure: test 18 'L1-dcache-load-miss:kp'running test 19 'mem:0:u'
running test 20 'mem:0:x:k'
running test 21 'mem:0:r:hp'
running test 22 'mem:0:w:up'
running test 23 'r1,syscalls:sys_enter_openat:k,1:1:hp'
FAILEDtests/parse-events.c:466wrong number of entries
Event test failure: test 23 'r1,syscalls:sys_enter_openat:k,1:1:hp'running test 24 'instructions:G'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 24 'instructions:G'running test 25 'instructions:H'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 25 'instructions:H'running test 26 'mem:0:rw'
running test 27 'mem:0:rw:kp'
running test 28 '{instructions:k,cycles:upp}'
FAILEDtests/parse-events.c:679wrong number of entries
Event test failure: test 28 '{instructions:k,cycles:upp}'running test 29 '{faults:k,cache-references}:u,cycles:k'
FAILEDtests/parse-events.c:721wrong number of entries
Event test failure: test 29 '{faults:k,cache-references}:u,cycles:k'running test 30 'group1{syscalls:sys_enter_openat:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u' FAILEDtests/parse-events.c:777wrong number of entries
Event test failure: test 30 'group1{syscalls:sys_enter_openat:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u'running test 31 '{cycles:u,instructions:kp}:p' FAILEDtests/parse-events.c:870wrong number of entries
Event test failure: test 31 '{cycles:u,instructions:kp}:p'running test 32 '{cycles,instructions}:G,{cycles:G,instructions:G},cycles'
FAILEDtests/parse-events.c:914wrong number of entries
Event test failure: test 32 '{cycles,instructions}:G,{cycles:G,instructions:G},cycles'running test 33 '*:*'
running test 34 '{cycles,cache-misses:G}:H'
FAILEDtests/parse-events.c:1000wrong number of entries
Event test failure: test 34 '{cycles,cache-misses:G}:H'running test 35 '{cycles,cache-misses:H}:G'
FAILEDtests/parse-events.c:1040wrong number of entries
Event test failure: test 35 '{cycles,cache-misses:H}:G'running test 36 '{cycles:G,cache-misses:H}:u'
FAILEDtests/parse-events.c:1080wrong number of entries
Event test failure: test 36 '{cycles:G,cache-misses:H}:u'running test 37 '{cycles:G,cache-misses:H}:uG'
FAILEDtests/parse-events.c:1120wrong number of entries
Event test failure: test 37 '{cycles:G,cache-misses:H}:uG'running test 38 '{cycles,cache-misses,branch-misses}:S'
FAILEDtests/parse-events.c:1160wrong number of entries
Event test failure: test 38 '{cycles,cache-misses,branch-misses}:S'running test 39 '{instructions,branch-misses}:Su'
FAILEDtests/parse-events.c:1213wrong number of entries
Event test failure: test 39 '{instructions,branch-misses}:Su'running test 40 'instructions:uDp'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 40 'instructions:uDp'running test 41 '{cycles,cache-misses,branch-misses}:D'
FAILEDtests/parse-events.c:1265wrong number of entries
Event test failure: test 41 '{cycles,cache-misses,branch-misses}:D'running test 42 'mem:0/1'
running test 43 'mem:0/2:w'
running test 44 'mem:0/4:rw:u'
running test 45 'instructions:I'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 45 'instructions:I'running test 46 'instructions:kIG'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 46 'instructions:kIG'running test 47 'task-clock:P,cycles'
FAILEDtests/parse-events.c:1382wrong number of entries
Event test failure: test 47 'task-clock:P,cycles'running test 48 'instructions/name=insn/'
running test 49 'r1234/name=rawpmu/'
running test 50 '4:0x6530160/name=numpmu/'
WARNING: event 'numpmu' not valid (bits 16-17,20,22 of config '6530160' not supported by kernel)!
running test 51 'L1-dcache-misses/name=cachepmu/'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 40
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
running test 52 'intel_pt//u'
running test 53 'cycles/name='COMPLEX_CYCLES_NAME:orig=cycles,desc=chip-clock-ticks'/Duk' running test 54 'cycles//u'
FAILEDtests/parse-events.c:1457wrong config
Event test failure: test 54 'cycles//u'running test 55 'cycles:k'
FAILEDtests/parse-events.c:1467wrong config
Event test failure: test 55 'cycles:k'running test 56 'instructions:uep'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 56 'instructions:uep'running test 57 '{cycles,cache-misses,branch-misses}:e' FAILEDtests/parse-events.c:1309wrong number of entries
Event test failure: test 57 '{cycles,cache-misses,branch-misses}:e'test child finished with -1 ---- end ----Quoted 1 line
Parse event definition strings subtest 1: FAILED!
Thanks! So "wrong number of entries" is the dominant failure and should be straightforward to make more PMU agnostic. Tests that are tied to a PMU like:
running test 52 'intel_pt//u'
succeed and it seems sad to lose coverage of them with this change. Overall, I think we should work to remove the hybrid test and make sure we get coverage in the general test. That means just skipping the test is the wrong thing to do.
Thanks,
IanQuoted 34 lines
Thanks,
Tinghao
Quoted 31 lines
--- tools/perf/tests/parse-events.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)tools/perf/tests/parse-events.c +4
-2@@ -2146,6 +2146,9 @@ static int test_events(const struct evlist_test *events, int cnt)static int test__events2 (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) { +if (perf_pmu__has_hybrid ()) +return TEST_SKIP ; +return test_events (test__events,ARRAY_SIZE (test__events)); }@@ -2421,7 +2424,7 @@ static int test__pmu_events_alias2(struct test_suite *test __maybe_unused,static struct test_case tests__parse_events[] = {TEST_CASE_REASON ("Test event parsing" , events2, -"permissions" ), +"permissions or hybrid" ),TEST_CASE_REASON ("Test parsing of \"hybrid\" CPU events" , hybrid,"not hybrid" ), --2.34.1
Quoted 84 lines · Ian Rogers
Quoted 36 lines · Kan
Quoted 26 lines · Ian Rogers
Quoted 17 lines · Tinghao
On 4/11/2023 9:50 PM, Ian Rogers wrote:
Quoted 10 lines · On Tue, Apr 11, 2023, 2:49 AM <[email protected]> wrote:
From: Tinghao Zhang <[email protected]>
The 'Test event parsing' test set under 'Parse event definion strings' does not apply to hybrid events. Since hybrid events have a dedicated test set 'Test parsing of hybrid CPU events', skip these tests on hybrid platforms.
Fixes 7741e03e808a ("perf test: Parse events break apart tests")Reported-by Yi Ammy <[email protected]>Signed-off-by Tinghao Zhang <[email protected]>This feels wrong and could make a different problem. Could you send the failure you are seeing? There is a verbose flag (-vv) that you can pass to perf test.
Quoted 7 lines
Thanks,
IanHi Ian,
I think these test cases are not designed for hybrid platforms. As the current implementation, if one general event type is used on hybrid platform, perf tool would generate seperate event for each pmu. But tests in 'Test event parsing' asserts only one evlist entry per event type. Another problem is that for
Quoted 7 lines
Hi Tinghao,
So you are right that the tests weren't designed for hybrid platforms because they didn't exist when the tests were created, but the bigger issue is what should perf be doing in these cases. On ARM's BIG.little version of "hybrid" there is a single PMU and generic events that work on both kind of core.
Quoted 4 lines
But we have different PMUs for Intel hybrid. The event list is quite different between e-core and p-core. There are some features which only available for one type of core. I don't think we can have a single PMU for Intel hybrid.
So we make the test and the expectation match.
Quoted 10 lines
Quoted 10 lines
But even with a single core we can get into
issues that an event can be defined for more than one PMU. For example, a GPU may have an instructions-retired count. In sysfs we'd have/sys/devices/my-gpuand/sys/devices/cpu. If the user asks for instructions-retired and has suitable permission, should the perf tool display instructions-retired for the GPU or just the cpu? My belief is that we should display my-gpu/instructions-retired/ and cpu/instructions-retired/. If the user wants to then limit to the CPU one, they can use the full PMU and event name cpu/instructions-retired/.Quoted 8 lines
Quoted 7 lines
Quoted 1 line
Why does this matter? Well if we have a common plan for how to deal
with events that match on more than 1 PMU then hybrid can just follow the plan and we can remove all of the hybrid code from the perf tool. Yes, the tests made an assumption there'd only be 1 of a kind of PMU for an event, so we need to change the test to assume that there will be at least 1, or that the number should match the number of PMUs or core PMUs.
How about the group? Using weak group?
No need, just make sure to call
parse_events__sort_events_and_fix_groups:
https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/util/parse-events.c?h=perf-tools-next#n2167Quoted 15 lines
Quoted 15 lines
Quoted 3 lines
PERF_TYPE_HARDWAREandPERF_TYPE_HW_CACHE, attr.config has a different format for hybrid events, so we see attr.config errors. Also, it is not applicable to put events from different PMUs to one group, but here some tests put general events in one group. As for the patch that first introduces hybrid test cases2541cb63ac0c("perf tests: Add hybrid cases for 'Parse event definition strings' test") , this test set is skipped. I think it's better if we skip this test set for hybrid pmus.So I think:
2541cb63ac0c("perf tests: Add hybrid cases for 'Parse event definition strings' test")
was added quickly in order that nothing be blocked, but with hindsight it regrettably didn't make a generic solution that could be used for a range of PMUs. You may be interested to see that the problem of being too CPU centric for perf is something reported by Intel's GPU team:
https://www.kernel.org/doc/html/next/gpu/i915.html#issues-hit-with-first-prototype-based-on-core-perf
I'd rather that the code were not hybrid or CPU specific, but everything was based off the PMU abstraction. For the attributes we'll need to improve the test to make it more generic to handle matching on multiple PMUs.Quoted 5 lines
The current "
test__events" doesn't generic enough. There is L1-dcache-load-miss which apparently CPU specific. Also ARCH specific events, e.g., kvm-s390,intel_pt. And groups.How should we make it generic? Any suggestions?
Quoted 9 lines
s390 uses #ifdefs,
intel_ptuses theintel_pt_validcall to enable the test - this returns false if theintel_ptPMU isn't detected. I have a fix for legacy events like L1-dcache-load-miss, the hybrid hack of having the parser run the parser inside itself is gross, we should just fix the parser. But given use of these legacy events is marginal/deprecated it isn't clear we need to support them on hybrid. Fwiw, I want to refactor the parsing code into a generic event toperf_event_attrlibrary that is more permissible than GPLv2, so I need to find some time to make it happen.
Yeah I think we need to refactor these test cases to base on the generic PMU abstractions.
And for test cases that are not generic, maybe we could create and move these to arch
specific test cases?
For the parser change Ian you suggest, don't know what's the impact on the current design?
Thanks,
Tinghao
Quoted 199 lines
Thanks,
IanQuoted 196 lines
Thanks,
KanQuoted 193 lines
Quoted 145 lines
Below is the perf test result on one Intel Meteor Lake platform
6: Parse event definition strings :
6.1: Test event parsing :
--- start ---
test child forked, pid 44965
running test 0 'syscalls:sys_enter_openat'
Using CPUID GenuineIntel-6-AA-2intel_ptdefault config: tsc,mtc,mtc_period=3,psb_period=3,pt,branch running test 1 'syscalls:*'
running test 2 'r1a'
FAILEDtests/parse-events.c:87wrong number of entries
Event test failure: test 2 'r1a'running test 3 '1:1'
running test 4 'instructions'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 4 'instructions'running test 5 'cycles/period=100000,config2/' FAILEDtests/parse-events.c:118wrong number of entries
Event test failure: test 5 'cycles/period=100000,config2/'running test 6 'faults' running test 7 'L1-dcache-load-miss'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 3
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
FAILEDtests/parse-events.c:152wrong config
Event test failure: test 7 'L1-dcache-load-miss'running test 8 'mem:0' running test 9 'mem:0:x'
running test 10 'mem:0:r'
running test 11 'mem:0:w'
running test 12 'syscalls:sys_enter_openat:k'
running test 13 'syscalls:*:u'
running test 14 'r1a:kp'
FAILEDtests/parse-events.c:87wrong number of entries Event test failure: test 14 'r1a:kp'running test 15 '1:1:hp' running test 16 'instructions:h'
FAILEDtests/parse-events.c:107wrong number of entries Event test failure: test 16 'instructions:h'running test 17 'faults:u' running test 18 'L1-dcache-load-miss:kp'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 39
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
FAILEDtests/parse-events.c:152wrong config
Event test failure: test 18 'L1-dcache-load-miss:kp'running test 19 'mem:0:u'
running test 20 'mem:0:x:k'
running test 21 'mem:0:r:hp'
running test 22 'mem:0:w:up'
running test 23 'r1,syscalls:sys_enter_openat:k,1:1:hp'
FAILEDtests/parse-events.c:466wrong number of entries
Event test failure: test 23 'r1,syscalls:sys_enter_openat:k,1:1:hp'running test 24 'instructions:G'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 24 'instructions:G'running test 25 'instructions:H'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 25 'instructions:H'running test 26 'mem:0:rw'
running test 27 'mem:0:rw:kp'
running test 28 '{instructions:k,cycles:upp}'
FAILEDtests/parse-events.c:679wrong number of entries
Event test failure: test 28 '{instructions:k,cycles:upp}'running test 29 '{faults:k,cache-references}:u,cycles:k'
FAILEDtests/parse-events.c:721wrong number of entries
Event test failure: test 29 '{faults:k,cache-references}:u,cycles:k'running test 30 'group1{syscalls:sys_enter_openat:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u' FAILEDtests/parse-events.c:777wrong number of entries
Event test failure: test 30 'group1{syscalls:sys_enter_openat:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u'running test 31 '{cycles:u,instructions:kp}:p' FAILEDtests/parse-events.c:870wrong number of entries
Event test failure: test 31 '{cycles:u,instructions:kp}:p'running test 32 '{cycles,instructions}:G,{cycles:G,instructions:G},cycles'
FAILEDtests/parse-events.c:914wrong number of entries
Event test failure: test 32 '{cycles,instructions}:G,{cycles:G,instructions:G},cycles'running test 33 '*:*'
running test 34 '{cycles,cache-misses:G}:H'
FAILEDtests/parse-events.c:1000wrong number of entries
Event test failure: test 34 '{cycles,cache-misses:G}:H'running test 35 '{cycles,cache-misses:H}:G'
FAILEDtests/parse-events.c:1040wrong number of entries
Event test failure: test 35 '{cycles,cache-misses:H}:G'running test 36 '{cycles:G,cache-misses:H}:u'
FAILEDtests/parse-events.c:1080wrong number of entries
Event test failure: test 36 '{cycles:G,cache-misses:H}:u'running test 37 '{cycles:G,cache-misses:H}:uG'
FAILEDtests/parse-events.c:1120wrong number of entries
Event test failure: test 37 '{cycles:G,cache-misses:H}:uG'running test 38 '{cycles,cache-misses,branch-misses}:S'
FAILEDtests/parse-events.c:1160wrong number of entries
Event test failure: test 38 '{cycles,cache-misses,branch-misses}:S'running test 39 '{instructions,branch-misses}:Su'
FAILEDtests/parse-events.c:1213wrong number of entries
Event test failure: test 39 '{instructions,branch-misses}:Su'running test 40 'instructions:uDp'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 40 'instructions:uDp'running test 41 '{cycles,cache-misses,branch-misses}:D'
FAILEDtests/parse-events.c:1265wrong number of entries
Event test failure: test 41 '{cycles,cache-misses,branch-misses}:D'running test 42 'mem:0/1'
running test 43 'mem:0/2:w'
running test 44 'mem:0/4:rw:u'
running test 45 'instructions:I'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 45 'instructions:I'running test 46 'instructions:kIG'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 46 'instructions:kIG'running test 47 'task-clock:P,cycles'
FAILEDtests/parse-events.c:1382wrong number of entries
Event test failure: test 47 'task-clock:P,cycles'running test 48 'instructions/name=insn/'
running test 49 'r1234/name=rawpmu/'
running test 50 '4:0x6530160/name=numpmu/'
WARNING: event 'numpmu' not valid (bits 16-17,20,22 of config '6530160' not supported by kernel)!
running test 51 'L1-dcache-misses/name=cachepmu/'
------------------------------------------------------------perf_event_attr:type 3 config 0x400010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8= 40
------------------------------------------------------------perf_event_attr:type 3 config 0x800010000 disabled 1------------------------------------------------------------
sys_perf_event_open: pid 0 cpu -1group_fd-1 flags0x8sys_perf_event_openfailed, error -2
running test 52 'intel_pt//u'
running test 53 'cycles/name='COMPLEX_CYCLES_NAME:orig=cycles,desc=chip-clock-ticks'/Duk' running test 54 'cycles//u'
FAILEDtests/parse-events.c:1457wrong config
Event test failure: test 54 'cycles//u'running test 55 'cycles:k'
FAILEDtests/parse-events.c:1467wrong config
Event test failure: test 55 'cycles:k'running test 56 'instructions:uep'
FAILEDtests/parse-events.c:107wrong number of entries
Event test failure: test 56 'instructions:uep'running test 57 '{cycles,cache-misses,branch-misses}:e' FAILEDtests/parse-events.c:1309wrong number of entries
Event test failure: test 57 '{cycles,cache-misses,branch-misses}:e'test child finished with -1 ---- end ----Quoted 1 line
Parse event definition strings subtest 1: FAILED!
Thanks! So "wrong number of entries" is the dominant failure and should be straightforward to make more PMU agnostic. Tests that are tied to a PMU like:
running test 52 'intel_pt//u'
succeed and it seems sad to lose coverage of them with this change. Overall, I think we should work to remove the hybrid test and make sure we get coverage in the general test. That means just skipping the test is the wrong thing to do.
Thanks,
IanQuoted 33 lines
Thanks,
Tinghao
Quoted 30 lines
--- tools/perf/tests/parse-events.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)tools/perf/tests/parse-events.c +4
-2@@ -2146,6 +2146,9 @@ static int test_events(const struct evlist_test *events, int cnt)static int test__events2 (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) { +if (perf_pmu__has_hybrid ()) +return TEST_SKIP ; +return test_events (test__events,ARRAY_SIZE (test__events)); }@@ -2421,7 +2424,7 @@ static int test__pmu_events_alias2(struct test_suite *test __maybe_unused,static struct test_case tests__parse_events[] = {TEST_CASE_REASON ("Test event parsing" , events2, -"permissions" ), +"permissions or hybrid" ),TEST_CASE_REASON ("Test parsing of \"hybrid\" CPU events" , hybrid,"not hybrid" ), --2.34.1