Skip to contentSkip to conversation

Some minor fixes for the perf tools json event code

PATCH In review Andi Kleen 4 participants 15 messages Oct 5, 2016 – Oct 17, 2016 perf events

lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

Fix some issues that turned up in recent testing.

-Andi

author
#1 lore raw

PATCH 1/3 perf, tools: Handle events including .c and .o

From: Andi Kleen <[email protected]>

This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.

Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.

Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.

---
 tools/perf/util/parse-events.l | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/parse-events.l +2 -2
index 9f43fda2570f..377147088a46 100644
@@ -183,8 +183,8 @@ modifier_bp	[rwx]{1,3}
 		}
 
 {event_pmu}	|
-{bpf_object}	|
-{bpf_source}	|
+({bpf_object}$)	|
+({bpf_source}$)	|
 {event}		{
 			BEGIN(INITIAL);
 			REWIND(1);
Signature · 1 line
2.5.5
#2 lore raw

Re: [PATCH 1/3] perf, tools: Handle events including .c and .o

Em Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:

Quoted 5 lines

From: Andi Kleen <[email protected]>

This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.

Quoted 7 lines

Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.

Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.

Wang, may I have your Acked-by, please?

- Arnaldo

Quoted 24 lines
---
 tools/perf/util/parse-events.l | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/parse-events.l +2 -2
index 9f43fda2570f..377147088a46 100644
@@ -183,8 +183,8 @@ modifier_bp	[rwx]{1,3}
 		}
 
 {event_pmu}	|
-{bpf_object}	|
-{bpf_source}	|
+({bpf_object}$)	|
+({bpf_source}$)	|
 {event}		{
 			BEGIN(INITIAL);
 			REWIND(1);
Signature · 1 line
2.5.5
author Tested-by Wang Nan
#3 lore raw

Re: [PATCH 1/3] perf, tools: Handle events including .c and .o

Quoted 10 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:

Quoted 9 lines

From: Andi Kleen <[email protected]>

This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.

Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.

Quoted 5 lines
Quoted 3 lines

Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.

Wang, may I have your Acked-by, please?

He acked it earlier here

https://patchwork.kernel.org/patch/9337721/

-Andi

#4 lore raw

Re: [PATCH 1/3] perf, tools: Handle events including .c and .o

Em Thu, Oct 06, 2016 at 09:55:05AM -0700, Andi Kleen escreveu:

Quoted 15 lines · Arnaldo Carvalho de Melo
Quoted 14 lines

Em Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:

Quoted 12 lines

From: Andi Kleen <[email protected]>

This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.

Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.

Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.

Quoted 7 lines
Quoted 1 line

Wang, may I have your Acked-by, please?

He acked it earlier here

https://patchwork.kernel.org/patch/9337721/

Ok, will add the example where it breaks in the commit message,

Thanks,

- Arnaldo

#5 lore raw

Re: [PATCH 1/3] perf, tools: Handle events including .c and .o

In reply to Andi Kleen

Em Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:

Quoted 5 lines

From: Andi Kleen <[email protected]>

This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.

Quoted 7 lines

Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.

Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.

So, I tried it with the example provided in the thread for a previous version of this patch (IIRC) and it still fails:

[acme@jouet linux]$ perf stat -e '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}' -a -I 1000
ERROR: problems with path {unc_p_clockticks,unc_p_power_state_occupancy.c: No such file or directory
event syntax error: '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}'
\___ Failed to load {unc_p_clockticks,unc_p_power_state_occupancy.c from source: Error when compiling BPF scriptlet

(add -v to see detail)
Run 'perf list' for a list of valid events

Usage: perf stat [<options>] [<command>]

-e, --event <event> event selector. use 'perf list' to list available events
[acme@jouet linux]$

And with another event that for sure is available on this machine:

[acme@jouet linux]$ perf stat -e '{uops_executed.core_cycles_ge_2}' -I 1000 usleep 10
ERROR: problems with path {uops_executed.c: No such file or directory
event syntax error: '{uops_executed.core_cycles_ge_2}'
\___ Failed to load {uops_executed.c from source: Error when compiling BPF scriptlet

(add -v to see detail)
Run 'perf list' for a list of valid events

Usage: perf stat [<options>] [<command>]

-e, --event <event> event selector. use 'perf list' to list available events
[acme@jouet linux]$

I thought this was due to the Makefile not noticing the change in the .l files, but I made sure I deleted the build dir and rebuilt from scratch, same problem.

- Arnaldo

Quoted 24 lines
---
 tools/perf/util/parse-events.l | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/parse-events.l +2 -2
index 9f43fda2570f..377147088a46 100644
@@ -183,8 +183,8 @@ modifier_bp	[rwx]{1,3}
 		}
 
 {event_pmu}	|
-{bpf_object}	|
-{bpf_source}	|
+({bpf_object}$)	|
+({bpf_source}$)	|
 {event}		{
 			BEGIN(INITIAL);
 			REWIND(1);
Signature · 1 line
2.5.5
#6 lore raw

Re: [PATCH 1/3] perf, tools: Handle events including .c and .o

Quoted 44 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 05, 2016 at 12:47:10PM -0700, Andi Kleen escreveu:

Quoted 5 lines

From: Andi Kleen <[email protected]>

This is a generic bug fix, but it helps with Sukadev's JSON event tree where such events can happen.

Quoted 7 lines

Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.

Fix the scanner to only match for .o or .c or .bpf at the end. This will prevent loading multiple BPF scripts separated with comma, but I assume this is acceptable.

So, I tried it with the example provided in the thread for a previous version of this patch (IIRC) and it still fails:

[acme@jouet linux]$ perf stat -e '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}' -a -I 1000
ERROR: problems with path {unc_p_clockticks,unc_p_power_state_occupancy.c: No such file or directory
event syntax error: '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}'
\___ Failed to load {unc_p_clockticks,unc_p_power_state_occupancy.c from source: Error when compiling BPF scriptlet

(add -v to see detail)
Run 'perf list' for a list of valid events

Usage: perf stat [<options>] [<command>]

-e, --event <event> event selector. use 'perf list' to list available events
[acme@jouet linux]$

And with another event that for sure is available on this machine:

[acme@jouet linux]$ perf stat -e '{uops_executed.core_cycles_ge_2}' -I 1000 usleep 10
ERROR: problems with path {uops_executed.c: No such file or directory
event syntax error: '{uops_executed.core_cycles_ge_2}'
\___ Failed to load {uops_executed.c from source: Error when compiling BPF scriptlet

(add -v to see detail)
Run 'perf list' for a list of valid events

Usage: perf stat [<options>] [<command>]

Quoted 9 lines

-e, --event <event> event selector. use 'perf list' to list available events
[acme@jouet linux]$

I thought this was due to the Makefile not noticing the change in the .l files, but I made sure I deleted the build dir and rebuilt from scratch, same problem.

- Arnaldo

Tested again, and thank you for giving us another chance for fixing this :)

The key problem here is not the ending '$' but the leading '{'. Flex's greedy maching policy makes this problem.

According to the design of parse-events.l, when it see something like '...{...}...', it first matches a 'group' in '<event>' scope, then rewind to INITIAL scope to match events in the group. In INITIAL scope, when it see a '{', flex consume this char and goes back to '<event>' scope to match next event. It works well before match BPF file path using unlimited '.*\.c' because '.*' will match the leading '{' in INITIAL scope without consuming it.

The simplest method for this problem is fixing the '.*' part: like what we define for 'event', don't match ',', '{' and '}'. Doesn't like 'event', '/' is required because this is a path.

Will post a patch for it. Please test it again.

Thank you.

#7 lore raw

PATCH 1/1 perf, tools: Handle events including .c and .o

In reply to Arnaldo Carvalho de Melo

This patch helps with Sukadev's JSON event tree where such events can happen.

Quoted 1 line

From Andi Kleen:

Any event inclduing a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel JSON events, which cannot be used.

This patch fixes this problem by forbidding BPF file patch containing '{', '}' and ',', make sure flex consumes the leading '{', instead of matcing it using a BPF file path.

Tested result:

  $ perf stat -e '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}' -a -I 1000
  invalid or unsupported event: '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}'
  Run 'perf list' for a list of valid events
  (as expected, interperted as event)
  $ perf stat -e 'aaa.c' -a -I 1000
  ERROR: problems with path aaa.c: No such file or directory
  (as expected, interperted as BPF source)
  $ perf stat -e 'aaa.ccc' -a -I 1000
  invalid or unsupported event: 'aaa.ccc'
  (as expected, interperted as event)
  $ perf stat -e '{aaa.c}' -a -I 1000
  ERROR: problems with path aaa.c: No such file or directory
  event syntax error: '{aaa.c}'
  <SKIP>
  (as expected, interperted as BPF source)
  $ perf stat -e '{cycles,aaa.c}' -a -I 1000
  ERROR: problems with path aaa.c: No such file or directory
  event syntax error: '{cycles,aaa.c}'
  (as expected, interperted as BPF source)
Cc Sukadev Bhattiprolu <[email protected]>
Cc Andi Kleen <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 tools/perf/util/parse-events.l | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/parse-events.l +2 -2
index 9f43fda..660fca0 100644
@@ -136,8 +136,8 @@ do {							\
 group		[^,{}/]*[{][^}]*[}][^,{}/]*
 event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
 event		[^,{}/]+
-bpf_object	.*\.(o|bpf)
-bpf_source	.*\.c
+bpf_object	[^,{}]+\.(o|bpf)
+bpf_source	[^,{}]+\.c
 
 num_dec		[0-9]+
 num_hex		0x[a-fA-F0-9]+
Signature · 1 line
1.8.3.4
#8 lore raw

[tip:perf/urgent] perf jevents: Handle events including .c and .o

perf jevents: Handle events including .c and .o
Commit-ID:  2d470b62fa24f8d0024e8d392d28814c287ee1f1
Gitweb:     http://git.kernel.org/tip/2d470b62fa24f8d0024e8d392d28814c287ee1f1
Author:     Wang Nan <[email protected]>
AuthorDate: Sat, 8 Oct 2016 04:16:25 +0000
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Mon, 17 Oct 2016 11:24:18 -0300

perf jevents: Handle events including .c and .o

This patch helps with Sukadev's vendor event tree where such events can happen.

Quoted 1 line

From Andi Kleen:

Any event including a .c/.o/.bpf currently triggers BPF compilation or loading and then an error. This can happen for some Intel vendor events, which cannot be used.

This patch fixes this problem by forbidding BPF file patch containing '{', '}' and ',', make sure flex consumes the leading '{', instead of matching it using a BPF file path.

Tested result:

  $ perf stat -e '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}' -a -I 1000
  invalid or unsupported event: '{unc_p_clockticks,unc_p_power_state_occupancy.cores_c0}'
  Run 'perf list' for a list of valid events
  (as expected, interperted as event)
  $ perf stat -e 'aaa.c' -a -I 1000
  ERROR: problems with path aaa.c: No such file or directory
  (as expected, interpreted as BPF source)
  $ perf stat -e 'aaa.ccc' -a -I 1000
  invalid or unsupported event: 'aaa.ccc'
  (as expected, interpreted as event)
  $ perf stat -e '{aaa.c}' -a -I 1000
  ERROR: problems with path aaa.c: No such file or directory
  event syntax error: '{aaa.c}'
  <SKIP>
  (as expected, interpreted as BPF source)
  $ perf stat -e '{cycles,aaa.c}' -a -I 1000
  ERROR: problems with path aaa.c: No such file or directory
  event syntax error: '{cycles,aaa.c}'
  (as expected, interpreted as BPF source)
---
 tools/perf/util/parse-events.l | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/parse-events.l +2 -2
index 9f43fda..660fca0 100644
@@ -136,8 +136,8 @@ do {							\
 group		[^,{}/]*[{][^}]*[}][^,{}/]*
 event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
 event		[^,{}/]+
-bpf_object	.*\.(o|bpf)
-bpf_source	.*\.c
+bpf_object	[^,{}]+\.(o|bpf)
+bpf_source	[^,{}]+\.c
 
 num_dec		[0-9]+
 num_hex		0x[a-fA-F0-9]+
author
#9 lore raw

PATCH 2/3 perf, tools: Handle completion of upper case events

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Vendor events are often specified in upper case. perf list outputs them in lower case. Handle this case in perf-completion.sh so that completion on the upper case events still works.

---
 tools/perf/perf-completion.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/perf/perf-completion.sh +5 -1
index 3ba80b2359cc..48e7a01b5c91 100644
@@ -161,7 +161,11 @@ __perf_main ()
 	# List possible events for -e option
 	elif [[ $prev == @("-e"|"--event") &&
 		$prev_skip_opts == @(record|stat|top) ]]; then
-		evts=$($cmd list --raw-dump)
+		# handle upper case events
+		case "$cur" in
+			[A-Z]*) evts=$($cmd list --raw-dump | tr a-z A-Z) ;;
+			*) evts=$($cmd list --raw-dump) ;;
+		esac
 		__perfcomp_colon "$evts" "$cur"
 	else
 		# List subcommands for perf commands
Signature · 1 line
2.5.5
#10 lore raw

Re: [PATCH 2/3] perf, tools: Handle completion of upper case events

Em Wed, Oct 05, 2016 at 12:47:11PM -0700, Andi Kleen escreveu:

Quoted 5 lines

From: Andi Kleen <[email protected]>

Vendor events are often specified in upper case. perf list outputs them in lower case. Handle this case in perf-completion.sh so that completion on the upper case events still works.

Humm, I just tried without your patch:

[root@jouet ~]# . ~acme/git/linux/tools/perf/perf-completion.sh [root@jouet ~]# perf stat -e cpu_clk_<TAB>

cpu_clk_thread_unhalted.one_thread_active  cpu_clk_unhalted.ref_tsc                   cpu_clk_unhalted.thread_any
cpu_clk_thread_unhalted.ref_xclk           cpu_clk_unhalted.ref_xclk                  cpu_clk_unhalted.thread_p
cpu_clk_thread_unhalted.ref_xclk_any       cpu_clk_unhalted.ref_xclk_any              cpu_clk_unhalted.thread_p_any
cpu_clk_unhalted.one_thread_active         cpu_clk_unhalted.thread                    

[root@jouet ~]# perf stat -e cpu_clk_

And then with it:

[root@jouet ~]# perf stat -e cpu<TAB>

And I get just beeps for the common case, i.e. lowercase, then if I try with uppercase it works:

[root@jouet ~]# perf stat -e CPU_CLK_

CPU_CLK_THREAD_UNHALTED.ONE_THREAD_ACTIVE  CPU_CLK_UNHALTED.REF_TSC                   CPU_CLK_UNHALTED.THREAD_ANY
CPU_CLK_THREAD_UNHALTED.REF_XCLK           CPU_CLK_UNHALTED.REF_XCLK                  CPU_CLK_UNHALTED.THREAD_P
CPU_CLK_THREAD_UNHALTED.REF_XCLK_ANY       CPU_CLK_UNHALTED.REF_XCLK_ANY              CPU_CLK_UNHALTED.THREAD_P_ANY
CPU_CLK_UNHALTED.ONE_THREAD_ACTIVE         CPU_CLK_UNHALTED.THREAD                    

[root@jouet ~]#

[root@jouet ~]# bash --version
GNU bash, version 4.3.42(1)-release (x86_64-redhat-linux-gnu) Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. [root@jouet ~]#

Quoted 24 lines
---
 tools/perf/perf-completion.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/perf/perf-completion.sh +5 -1
index 3ba80b2359cc..48e7a01b5c91 100644
@@ -161,7 +161,11 @@ __perf_main ()
 	# List possible events for -e option
 	elif [[ $prev == @("-e"|"--event") &&
 		$prev_skip_opts == @(record|stat|top) ]]; then
-		evts=$($cmd list --raw-dump)
+		# handle upper case events
+		case "$cur" in
+			[A-Z]*) evts=$($cmd list --raw-dump | tr a-z A-Z) ;;
+			*) evts=$($cmd list --raw-dump) ;;
+		esac
 		__perfcomp_colon "$evts" "$cur"
 	else
 		# List subcommands for perf commands
Signature · 1 line
2.5.5
author
#11 lore raw

Re: [PATCH 2/3] perf, tools: Handle completion of upper case events

Quoted 8 lines

[root@jouet ~]# bash --version
GNU bash, version 4.3.42(1)-release (x86_64-redhat-linux-gnu) Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. [root@jouet ~]#

Probably a locale issue. Let me send a new patch.

-Andi

#12 lore raw

Re: [PATCH 2/3] perf, tools: Handle completion of upper case events

Em Wed, Oct 05, 2016 at 03:18:05PM -0700, Andi Kleen escreveu:

Quoted 10 lines
Quoted 8 lines

[root@jouet ~]# bash --version
GNU bash, version 4.3.42(1)-release (x86_64-redhat-linux-gnu) Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. [root@jouet ~]#

Probably a locale issue. Let me send a new patch.

[root@jouet ~]# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
[root@jouet ~]#

- Arnaldo

author
#13 lore raw

PATCH 3/3 perf, tools: Fix Intel fixed counter conversions

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Intel fixed counters are special cases in the JSON conversion process because their decoding differs between perf and the event files. Add some missing entries in the conversion table.

---
 tools/perf/pmu-events/jevents.c | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/pmu-events/jevents.c +2 -0
index 79c2133bc534..41611d7f9873 100644
@@ -312,6 +312,8 @@ static struct fixed {
 	const char *event;
 } fixed[] = {
 	{ "inst_retired.any", "event=0xc0" },
+	{ "inst_retired.any_p", "event=0xc0" },
+	{ "cpu_clk_unhalted.ref", "event=0x0,umask=0x03" },
 	{ "cpu_clk_unhalted.thread", "event=0x3c" },
 	{ "cpu_clk_unhalted.thread_any", "event=0x3c,any=1" },
 	{ NULL, NULL},
Signature · 1 line
2.5.5
#14 lore raw

[tip:perf/urgent] perf jevents: Fix Intel JSON fixed counter conversions

perf jevents: Fix Intel JSON fixed counter conversions
Commit-ID:  72c6ff2583fba824dc38c0ce87b838631cdb8294
Gitweb:     http://git.kernel.org/tip/72c6ff2583fba824dc38c0ce87b838631cdb8294
Author:     Andi Kleen <[email protected]>
AuthorDate: Wed, 5 Oct 2016 12:47:12 -0700
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 5 Oct 2016 18:41:06 -0300

perf jevents: Fix Intel JSON fixed counter conversions

Intel fixed counters are special cases in the JSON conversion process because their decoding differs between perf and the event files. Add some missing entries in the conversion table.

---
 tools/perf/pmu-events/jevents.c | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/pmu-events/jevents.c +2 -0
index 79c2133..41611d7 100644
@@ -312,6 +312,8 @@ static struct fixed {
 	const char *event;
 } fixed[] = {
 	{ "inst_retired.any", "event=0xc0" },
+	{ "inst_retired.any_p", "event=0xc0" },
+	{ "cpu_clk_unhalted.ref", "event=0x0,umask=0x03" },
 	{ "cpu_clk_unhalted.thread", "event=0x3c" },
 	{ "cpu_clk_unhalted.thread_any", "event=0x3c,any=1" },
 	{ NULL, NULL},