Some minor fixes for the perf tools json event code
Fix some issues that turned up in recent testing.
-Andi
PATCH 1/3 perf, tools: Handle events including .c and .o
From: Andi Kleen <[email protected]>
This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.
Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.
Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.
--- tools/perf/util/parse-events.l | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/parse-events.l +2 -2
@@ -183,8 +183,8 @@ modifier_bp [rwx]{1,3} } {event_pmu} | -{bpf_object} | -{bpf_source} | +({bpf_object}$) | +({bpf_source}$) | {event} { BEGIN(INITIAL); REWIND(1);
Signature · 1 line
2.5.5
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 1/3] perf, tools: Handle events including .c and .o
Em Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:
Quoted 5 lines
From: Andi Kleen <[email protected]>
This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.
Quoted 7 lines
Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.
Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.
Wang, may I have your Acked-by, please?
- Arnaldo
Quoted 24 lines
--- tools/perf/util/parse-events.l | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/perf/util/parse-events.l +2
-2@@ -183,8 +183,8 @@ modifier_bp [rwx]{1,3} } {event_pmu} | -{bpf_object} | -{bpf_source} | +({bpf_object}$) | +({bpf_source}$) | {event} { BEGIN(INITIAL); REWIND(1);Signature · 1 line
2.5.5
Andi Kleen
Tested-by Wang NanRe: [PATCH 1/3] perf, tools: Handle events including .c and .o
Quoted 10 lines · Arnaldo Carvalho de Melo
Em Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:
Quoted 9 lines
From: Andi Kleen <[email protected]>
This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.
Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.
Quoted 5 lines
Quoted 3 lines
Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.
Wang, may I have your Acked-by, please?
He acked it earlier here
https://patchwork.kernel.org/patch/9337721/
-Andi
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 1/3] perf, tools: Handle events including .c and .o
Em Thu, Oct 06, 2016 at 09:55:05AM -0700, Andi Kleen escreveu:
Quoted 15 lines · Arnaldo Carvalho de Melo
Quoted 14 lines
Em Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:
Quoted 12 lines
From: Andi Kleen <[email protected]>
This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.
Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.
Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.
Quoted 7 lines
Quoted 1 line
Wang, may I have your Acked-by, please?
He acked it earlier here
https://patchwork.kernel.org/patch/9337721/
Tested-by Wang Nan <[email protected]>
Ok, will add the example where it breaks in the commit message,
Thanks,
- Arnaldo
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 1/3] perf, tools: Handle events including .c and .o
In reply to Andi KleenEm Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:
Quoted 5 lines
From: Andi Kleen <[email protected]>
This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.
Quoted 7 lines
Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.
Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.
So, I tried it with the example provided in the thread for a previous version of this patch (IIRC) and it still fails:
[acme@jouet linux]$ perf stat -e '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}' -a -I 1000
ERROR: problems with path {unc_p_clockticks,unc_p_power_state_occupancy.c: No such file or directory
event syntax error: '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}'
\___ Failed to load {unc_p_clockticks,unc_p_power_state_occupancy.c from source: Error when compiling BPF scriptlet
(add -v to see detail)
Run 'perf list' for a list of valid events
Usage: perf stat [<options>] [<command>]
-e, --event <event> event selector. use 'perf list' to list available events
[acme@jouet linux]$
And with another event that for sure is available on this machine:
[acme@jouet linux]$ perf stat -e '{uops_executed.core_cycles_ge_2}' -I 1000 usleep 10
ERROR: problems with path {uops_executed.c: No such file or directory
event syntax error: '{uops_executed.core_cycles_ge_2}'
\___ Failed to load {uops_executed.c from source: Error when compiling BPF scriptlet
(add -v to see detail)
Run 'perf list' for a list of valid events
Usage: perf stat [<options>] [<command>]
-e, --event <event> event selector. use 'perf list' to list available events
[acme@jouet linux]$
I thought this was due to the Makefile not noticing the change in the .l files, but I made sure I deleted the build dir and rebuilt from scratch, same problem.
- Arnaldo
Quoted 24 lines
--- tools/perf/util/parse-events.l | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/perf/util/parse-events.l +2
-2@@ -183,8 +183,8 @@ modifier_bp [rwx]{1,3} } {event_pmu} | -{bpf_object} | -{bpf_source} | +({bpf_object}$) | +({bpf_source}$) | {event} { BEGIN(INITIAL); REWIND(1);Signature · 1 line
2.5.5
Re: [PATCH 1/3] perf, tools: Handle events including .c and .o
Quoted 44 lines · Arnaldo Carvalho de Melo
Em Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:
Quoted 5 lines
From: Andi Kleen <[email protected]>
This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.
Quoted 7 lines
Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.
Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.
So, I tried it with the example provided in the thread for a previous version of this patch (IIRC) and it still fails:
[acme@jouet linux]$ perf stat -e '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}' -a -I 1000
ERROR: problems with path {unc_p_clockticks,unc_p_power_state_occupancy.c: No such file or directory
event syntax error: '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}'
\___ Failed to load {unc_p_clockticks,unc_p_power_state_occupancy.c from source: Error when compiling BPF scriptlet(add -v to see detail)
Run 'perf list' for a list of valid eventsUsage: perf stat [<options>] [<command>]
-e, --event <event> event selector. use 'perf list' to list available events
[acme@jouet linux]$And with another event that for sure is available on this machine:
[acme@jouet linux]$ perf stat -e '{uops_executed.core_cycles_ge_2}' -I 1000 usleep 10
ERROR: problems with path {uops_executed.c: No such file or directory
event syntax error: '{uops_executed.core_cycles_ge_2}'
\___ Failed to load {uops_executed.cfrom source: Error when compiling BPF scriptlet(add -v to see detail)
Run 'perf list' for a list of valid eventsUsage: perf stat [<options>] [<command>]
Quoted 9 lines
-e, --event <event> event selector. use 'perf list' to list available events
[acme@jouet linux]$I thought this was due to the
Makefilenot noticing the change in the .l files, but I made sure I deleted the build dir and rebuilt from scratch, same problem.- Arnaldo
Tested again, and thank you for giving us another chance for fixing this :)
The key problem here is not the ending '$' but the leading '{'. Flex's greedy maching policy makes this problem.
According to the design of parse-events.l, when it see something like '...{...}...', it first matches a 'group' in '<event>' scope, then rewind to INITIAL scope to match events in the group. In INITIAL scope, when it see a '{', flex consume this char and goes back to '<event>' scope to match next event. It works well before match BPF file path using unlimited '.*\.c' because '.*' will match the leading '{' in INITIAL scope without consuming it.
The simplest method for this problem is fixing the '.*' part: like what we define for 'event', don't match ',', '{' and '}'. Doesn't like 'event', '/' is required because this is a path.
Will post a patch for it. Please test it again.
Thank you.
PATCH 1/1 perf, tools: Handle events including .c and .o
In reply to Arnaldo Carvalho de MeloThis patch helps with Sukadev's JSON event tree where such events can happen.
Quoted 1 line
From Andi Kleen:
Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.
This patch fixes this problem by forbidding BPF file patch containing '{', '}' and ',', make sure flex consumes the leading '{', instead of matcing it using a BPF file path.
Tested result:
$ perf stat -e '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}' -a -I 1000
invalid or unsupported event: '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}'
Run 'perf list' for a list of valid events
(as expected, interperted as event)$ perf stat -e 'aaa.c' -a -I 1000 ERROR: problems with path aaa.c: No such file or directory (as expected, interperted as BPF source)
$ perf stat -e 'aaa.ccc' -a -I 1000 invalid or unsupported event: 'aaa.ccc' (as expected, interperted as event)
$ perf stat -e '{aaa.c}' -a -I 1000
ERROR: problems with path aaa.c: No such file or directory
event syntax error: '{aaa.c}'
<SKIP>
(as expected, interperted as BPF source) $ perf stat -e '{cycles,aaa.c}' -a -I 1000
ERROR: problems with path aaa.c: No such file or directory
event syntax error: '{cycles,aaa.c}'
(as expected, interperted as BPF source)--- tools/perf/util/parse-events.l | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/parse-events.l +2 -2
@@ -136,8 +136,8 @@ do { \ group [^,{}/]*[{][^}]*[}][^,{}/]* event_pmu [^,{}/]+[/][^/]*[/][^,{}/]* event [^,{}/]+ -bpf_object .*\.(o|bpf) -bpf_source .*\.c +bpf_object [^,{}]+\.(o|bpf) +bpf_source [^,{}]+\.c num_dec [0-9]+ num_hex 0x[a-fA-F0-9]+
Signature · 1 line
1.8.3.4
tip-bot for Wang Nan
merge bot[tip:perf/urgent] perf jevents: Handle events including .c and .o
perf jevents: Handle events including .c and .o
Commit-ID: 2d470b62fa24f8d0024e8d392d28814c287ee1f1 Gitweb: http://git.kernel.org/tip/2d470b62fa24f8d0024e8d392d28814c287ee1f1 Author: Wang Nan <[email protected]> AuthorDate: Sat, 8 Oct 2016 04:16:25 +0000 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Mon, 17 Oct 2016 11:24:18 -0300
perf jevents: Handle events including .c and .o
This patch helps with Sukadev's vendor event tree where such events can happen.
Quoted 1 line
From Andi Kleen:
Any event including a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel vendor events, which cannot be used.
This patch fixes this problem by forbidding BPF file patch containing '{', '}' and ',', make sure flex consumes the leading '{', instead of matching it using a BPF file path.
Tested result:
$ perf stat -e '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}' -a -I 1000
invalid or unsupported event: '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}'
Run 'perf list' for a list of valid events
(as expected, interperted as event)$ perf stat -e 'aaa.c' -a -I 1000 ERROR: problems with path aaa.c: No such file or directory (as expected, interpreted as BPF source)
$ perf stat -e 'aaa.ccc' -a -I 1000 invalid or unsupported event: 'aaa.ccc' (as expected, interpreted as event)
$ perf stat -e '{aaa.c}' -a -I 1000
ERROR: problems with path aaa.c: No such file or directory
event syntax error: '{aaa.c}'
<SKIP>
(as expected, interpreted as BPF source) $ perf stat -e '{cycles,aaa.c}' -a -I 1000
ERROR: problems with path aaa.c: No such file or directory
event syntax error: '{cycles,aaa.c}'
(as expected, interpreted as BPF source)--- tools/perf/util/parse-events.l | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/parse-events.l +2 -2
@@ -136,8 +136,8 @@ do { \ group [^,{}/]*[{][^}]*[}][^,{}/]* event_pmu [^,{}/]+[/][^/]*[/][^,{}/]* event [^,{}/]+ -bpf_object .*\.(o|bpf) -bpf_source .*\.c +bpf_object [^,{}]+\.(o|bpf) +bpf_source [^,{}]+\.c num_dec [0-9]+ num_hex 0x[a-fA-F0-9]+
PATCH 2/3 perf, tools: Handle completion of upper case events
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Vendor events are often specified in upper case. perf list outputs them in lower case. Handle this case in perf-completion.sh so that completion on the upper case events still works.
--- tools/perf/perf-completion.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
tools/perf/perf-completion.sh +5 -1
@@ -161,7 +161,11 @@ __perf_main () # List possible events for -e option elif [[ $prev == @("-e"|"--event") && $prev_skip_opts == @(record|stat|top) ]]; then - evts=$($cmd list --raw-dump) + # handle upper case events + case "$cur" in + [A-Z]*) evts=$($cmd list --raw-dump | tr a-z A-Z) ;; + *) evts=$($cmd list --raw-dump) ;; + esac __perfcomp_colon "$evts" "$cur" else # List subcommands for perf commands
Signature · 1 line
2.5.5
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 2/3] perf, tools: Handle completion of upper case events
Em Wed, Oct 05, 2016 at 12:47:11PM -0700, Andi Kleen escreveu:
Quoted 5 lines
From: Andi Kleen <[email protected]>
Vendor events are often specified in upper case. perf list outputs them in lower case. Handle this case in
perf-completion.shso that completion on the upper case events still works.
Humm, I just tried without your patch:
[root@jouet ~]# . ~acme/git/linux/tools/perf/perf-completion.sh [root@jouet ~]# perf stat -e cpu_clk_<TAB>
cpu_clk_thread_unhalted.one_thread_active cpu_clk_unhalted.ref_tsc cpu_clk_unhalted.thread_any cpu_clk_thread_unhalted.ref_xclk cpu_clk_unhalted.ref_xclk cpu_clk_unhalted.thread_p cpu_clk_thread_unhalted.ref_xclk_any cpu_clk_unhalted.ref_xclk_any cpu_clk_unhalted.thread_p_any cpu_clk_unhalted.one_thread_active cpu_clk_unhalted.thread
[root@jouet ~]# perf stat -e cpu_clk_
And then with it:
[root@jouet ~]# perf stat -e cpu<TAB>
And I get just beeps for the common case, i.e. lowercase, then if I try with uppercase it works:
[root@jouet ~]# perf stat -e CPU_CLK_
CPU_CLK_THREAD_UNHALTED.ONE_THREAD_ACTIVE CPU_CLK_UNHALTED.REF_TSC CPU_CLK_UNHALTED.THREAD_ANY CPU_CLK_THREAD_UNHALTED.REF_XCLK CPU_CLK_UNHALTED.REF_XCLK CPU_CLK_UNHALTED.THREAD_P CPU_CLK_THREAD_UNHALTED.REF_XCLK_ANY CPU_CLK_UNHALTED.REF_XCLK_ANY CPU_CLK_UNHALTED.THREAD_P_ANY CPU_CLK_UNHALTED.ONE_THREAD_ACTIVE CPU_CLK_UNHALTED.THREAD
[root@jouet ~]#
[root@jouet ~]# bash --version
GNU bash, version 4.3.42(1)-release (x86_64-redhat-linux-gnu) Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. [root@jouet ~]#
Quoted 24 lines
Signed-off-by Andi Kleen <[email protected]>--- tools/perf/perf-completion.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)tools/perf/perf-completion.sh +5
-1@@ -161,7 +161,11 @@ __perf_main () # List possible events for -e option elif [[ $prev == @("-e"|"--event") && $prev_skip_opts == @(record|stat|top) ]]; then - evts=$($cmd list --raw-dump) + # handle upper case events + case "$cur" in + [A-Z]*) evts=$($cmd list --raw-dump | tr a-z A-Z) ;; + *) evts=$($cmd list --raw-dump) ;; + esac __perfcomp_colon "$evts" "$cur" else # List subcommands for perf commandsSignature · 1 line
2.5.5
Re: [PATCH 2/3] perf, tools: Handle completion of upper case events
Quoted 8 lines
[root@jouet ~]# bash --version
GNU bash, version 4.3.42(1)-release (x86_64-redhat-linux-gnu) Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. [root@jouet ~]#
Probably a locale issue. Let me send a new patch.
-Andi
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 2/3] perf, tools: Handle completion of upper case events
Em Wed, Oct 05, 2016 at 03:18:05PM -0700, Andi Kleen escreveu:
Quoted 10 lines
Quoted 8 lines
[root@jouet ~]# bash --version
GNU bash, version 4.3.42(1)-release (x86_64-redhat-linux-gnu) Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. [root@jouet ~]#
Probably a locale issue. Let me send a new patch.
[root@jouet ~]# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
[root@jouet ~]#
- Arnaldo
PATCH 3/3 perf, tools: Fix Intel fixed counter conversions
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Intel fixed counters are special cases in the JSON conversion process because their decoding differs between perf and the event files. Add some missing entries in the conversion table.
--- tools/perf/pmu-events/jevents.c | 2 ++ 1 file changed, 2 insertions(+)
tools/perf/pmu-events/jevents.c +2 -0
@@ -312,6 +312,8 @@ static struct fixed {const char *event; } fixed[] = { { "inst_retired.any", "event=0xc0" }, + { "inst_retired.any_p", "event=0xc0" }, + { "cpu_clk_unhalted.ref", "event=0x0,umask=0x03" }, { "cpu_clk_unhalted.thread", "event=0x3c" }, { "cpu_clk_unhalted.thread_any", "event=0x3c,any=1" }, { NULL, NULL},
Signature · 1 line
2.5.5
tip-bot for Andi Kleen
merge bot[tip:perf/urgent] perf jevents: Fix Intel JSON fixed counter conversions
perf jevents: Fix Intel JSON fixed counter conversions
Commit-ID: 72c6ff2583fba824dc38c0ce87b838631cdb8294 Gitweb: http://git.kernel.org/tip/72c6ff2583fba824dc38c0ce87b838631cdb8294 Author: Andi Kleen <[email protected]> AuthorDate: Wed, 5 Oct 2016 12:47:12 -0700 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Wed, 5 Oct 2016 18:41:06 -0300
perf jevents: Fix Intel JSON fixed counter conversions
Intel fixed counters are special cases in the JSON conversion process because their decoding differs between perf and the event files. Add some missing entries in the conversion table.
--- tools/perf/pmu-events/jevents.c | 2 ++ 1 file changed, 2 insertions(+)
tools/perf/pmu-events/jevents.c +2 -0
@@ -312,6 +312,8 @@ static struct fixed {const char *event; } fixed[] = { { "inst_retired.any", "event=0xc0" }, + { "inst_retired.any_p", "event=0xc0" }, + { "cpu_clk_unhalted.ref", "event=0x0,umask=0x03" }, { "cpu_clk_unhalted.thread", "event=0x3c" }, { "cpu_clk_unhalted.thread_any", "event=0x3c,any=1" }, { NULL, NULL},