perf script: add flamegraph.py script
PATCH 1/1 perf script: add flamegraph.py script
This script works in tandem with d3-flame-graph to generate flame graphs from perf. It supports two output formats: JSON and HTML (the default). The HTML format will look for a standalone d3-flame-graph template file in /usr/share/d3-flame-graph/d3-flamegraph-base.html and fill in the collected stacks.
Usage:
perf record -a -g -F 99 sleep 60
perf script report flamegraph
Combined:
perf script flamegraph -a -F 99 sleep 60
---
Package for d3-flame-graph is currently under review, in the meantime you can download the template from:
https://cdn.jsdelivr.net/npm/[email protected]/dist/templates/d3-flamegraph-base.html
and move it to /usr/share/d3-flame-graph/d3-flamegraph-base.html
.../perf/scripts/python/bin/flamegraph-record | 2 + .../perf/scripts/python/bin/flamegraph-report | 3 + tools/perf/scripts/python/flamegraph.py | 126 ++++++++++++++++++ 3 files changed, 131 insertions(+) create mode 100755 tools/perf/scripts/python/bin/flamegraph-record create mode 100755 tools/perf/scripts/python/bin/flamegraph-report create mode 100755 tools/perf/scripts/python/flamegraph.py
tools/perf/scripts/python/bin/flamegraph-record new file +2 -0
@@ -0,0 +1,2 @@ +#!/usr/bin/sh +perf record -g "$@"
tools/perf/scripts/python/bin/flamegraph-report new file +3 -0
@@ -0,0 +1,3 @@ +#!/usr/bin/sh +# description: create flame graphs +perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
tools/perf/scripts/python/flamegraph.py new file +126 -0
126 lines not on this page · raw message
Signature · 1 line
2.25.1
Quoted 10 lines · Andreas Gerstmayr
This script works in tandem with d3-flame-graph to generate flame graphs from perf. It supports two output formats: JSON and HTML (the default). The HTML format will look for a standalone d3-flame-graph template file in
/usr/share/d3-flame-graph/d3-flamegraph-base.htmland fill in the collected stacks.Usage:
perf record -a -g -F 99 sleep 60
perf script report flamegraph
On Ubuntu 19.10, where python 2.7 is still the default, I get:
$ perf script report flamegraph
File "/usr/libexec/perf-core/scripts/python/flamegraph.py", line 46
print(f"Flame Graph template {self.args.template} does not " +
^
SyntaxError: invalid syntax
Error running python script /usr/libexec/perf-core/scripts/python/flamegraph.pyInstalling libpython3-dev doesn't help.
$ perf script -s lang
Scripting language extensions (used in perf script -s [spec:]script.[spec]):
Perl [Perl] pl [Perl] Python [Python] py [Python]
Should there be a python3 in that list?
Thanks,
Kim
Quoted 10 lines · Kim Phillips
On Ubuntu 19.10, where python 2.7 is still the default, I get:
$ perf script report flamegraph File "/usr/libexec/perf-core/scripts/python/flamegraph.py", line 46 print(f"Flame Graph template {self.args.template} does not " + ^ SyntaxError: invalid syntax Error running python script /usr/libexec/perf-core/scripts/python/flamegraph.pyInstalling libpython3-dev doesn't help.
Hmm, I was hoping that I can drop support for Python 2 in 2020 ;) (it's officially EOL since Jan 1, 2020)
The Ubuntu 18.04 release notes mention that "Python 2 is no longer installed by default. Python 3 has been updated to 3.6. This is the last LTS release to include Python 2 in main." (https://wiki.ubuntu.com/BionicBeaver/ReleaseNotes) - so imho it should be fine to drop Python 2 support.
I tested it with a Ubuntu VM, and by default the Python bindings aren't enabled in perf (see https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1707875).
But you can compile perf and select Python 3:
$ make -j2 PYTHON=python3
in the perf source directory (libpython3-dev must be installed).
Does this work for you?
Cheers,
Andreas
Quoted 26 lines · Andreas Gerstmayr
Quoted 4 lines · Kim Phillips
On Ubuntu 19.10, where python 2.7 is still the default, I get:
$ perf script report flamegraph File "/usr/libexec/perf-core/scripts/python/flamegraph.py", line46
Quoted 4 lines
print(f"Flame Graph template {self.args.template} does not " +
^
SyntaxError: invalid syntax
Error running python script
/usr/libexec/perf-core/scripts/python/flamegraph.pyQuoted 2 lines
Installing libpython3-dev doesn't help.
Hmm, I was hoping that I can drop support for Python 2 in 2020 ;) (it's
officially EOL since Jan 1, 2020)
The Ubuntu 18.04 release notes mention that "Python 2 is no longer installed by default. Python 3 has been updated to 3.6. This is the last
LTS release to include Python 2 in main."
(https://wiki.ubuntu.com/BionicBeaver/ReleaseNotes) - so imho it shouldbe fine to drop Python 2 support.
Quoted 9 lines
I tested it with a Ubuntu VM, and by default the Python bindings aren't
enabled in perf (see
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1707875).But you can compile perf and select Python 3:
$ make -j2 PYTHON=python3
I plan to make this the default soon.
- Arnaldo
Quoted 8 lines
in the perf source directory (libpython3-dev must be installed).
Does this work for you?
Cheers,
Andreas
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Quoted 18 lines · Andreas Gerstmayr
Quoted 10 lines · Kim Phillips
On Ubuntu 19.10, where python 2.7 is still the default, I get:
$ perf script report flamegraph File "/usr/libexec/perf-core/scripts/python/flamegraph.py", line 46 print(f"Flame Graph template {self.args.template} does not " + ^ SyntaxError: invalid syntax Error running python script /usr/libexec/perf-core/scripts/python/flamegraph.pyInstalling libpython3-dev doesn't help.
Hmm, I was hoping that I can drop support for Python 2 in 2020 ;) (it's officially EOL since Jan 1, 2020)
The Ubuntu 18.04 release notes mention that "Python 2 is no longer installed by default. Python 3 has been updated to 3.6. This is the last LTS release to include Python 2 in main." (https://wiki.ubuntu.com/BionicBeaver/ReleaseNotes) - so imho it should be fine to drop Python 2 support.
I tested it with a Ubuntu VM, and by default the Python bindings aren't enabled in perf (see https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1707875).
Quoted 8 lines
But you can compile perf and select Python 3:
$ make -j2 PYTHON=python3in the perf source directory (libpython3-dev must be installed).
Does this work for you?
Not on Ubuntu 18.04.4 LTS, but it does on 19.10.
On 19.10 however, when specifying dwarf on the record, e.g.:
sudo perf record -a -g -C2,4 --call-graph=dwarf -- sleep 10
I now get a SIGSEGV when executing perf script report flamegraph.
Here's a trace:
#0 0x000055555590a9b2 in regs_map (regs=0x7fffffffbfc8, mask=16715775,
bf=0x7fffffffba60 "", size=512) at util/scripting-engines/trace-event-python.c:696
#1 0x000055555590ab03 in set_regs_in_dict (dict=0x7ffff61dd500, sample=0x7fffffffbf20,
evsel=0x555555d7a700) at util/scripting-engines/trace-event-python.c:718
#2 0x000055555590af1f in get_perf_sample_dict (sample=0x7fffffffbf20,
evsel=0x555555d7a700, al=0x7fffffffbdd0, callchain=0x7ffff625b780)
at util/scripting-engines/trace-event-python.c:787
#3 0x000055555590ce3e in python_process_general_event (sample=0x7fffffffbf20,
evsel=0x555555d7a700, al=0x7fffffffbdd0)
at util/scripting-engines/trace-event-python.c:1301
#4 0x000055555590cf94 in python_process_event (event=0x7ffff60b0a48,
sample=0x7fffffffbf20, evsel=0x555555d7a700, al=0x7fffffffbdd0)
at util/scripting-engines/trace-event-python.c:1328
#5 0x000055555577375c in process_sample_event (tool=0x7fffffffcf30,
event=0x7ffff60b0a48, sample=0x7fffffffbf20, evsel=0x555555d7a700,
machine=0x555555d73168) at builtin-script.c:2072
#6 0x000055555585f3d9 in perf_evlist__deliver_sample (evlist=0x555555d79c60,
tool=0x7fffffffcf30, event=0x7ffff60b0a48, sample=0x7fffffffbf20,
evsel=0x555555d7a700, machine=0x555555d73168) at util/session.c:1389
#7 0x000055555585f588 in machines__deliver_event (machines=0x555555d73168,
evlist=0x555555d79c60, event=0x7ffff60b0a48, sample=0x7fffffffbf20,
tool=0x7fffffffcf30, file_offset=3037768) at util/session.c:1426
#8 0x000055555585fa32 in perf_session__deliver_event (session=0x555555d72fe0,
event=0x7ffff60b0a48, tool=0x7fffffffcf30, file_offset=3037768)
at util/session.c:1499
#9 0x000055555585bf5e in ordered_events__deliver_event (oe=0x555555d79b20,
event=0x555556446588) at util/session.c:183#10 0x0000555555864010 in do_flush (oe=0x555555d79b20, show_progress=false)
at util/ordered-events.c:244
#11 0x000055555586435f in __ordered_events__flush (oe=0x555555d79b20,
how=OE_FLUSH__ROUND, timestamp=0) at util/ordered-events.c:323
#12 0x0000555555864447 in ordered_events__flush (oe=0x555555d79b20, how=OE_FLUSH__ROUND)
at util/ordered-events.c:341
#13 0x000055555585e2b1 in process_finished_round (tool=0x7fffffffcf30,
event=0x7ffff60ec040, oe=0x555555d79b20) at util/session.c:997
#14 0x000055555585fcea in perf_session__process_user_event (session=0x555555d72fe0,
event=0x7ffff60ec040, file_offset=3280960) at util/session.c:1546
#15 0x000055555586055d in perf_session__process_event (session=0x555555d72fe0,
event=0x7ffff60ec040, file_offset=3280960) at util/session.c:1706
#16 0x0000555555861973 in process_simple (session=0x555555d72fe0, event=0x7ffff60ec040,
file_offset=3280960) at util/session.c:2202
#17 0x0000555555861792 in reader__process_events (rd=0x7fffffffcd70,
session=0x555555d72fe0, prog=0x7fffffffcd90) at util/session.c:2168
#18 0x0000555555861a68 in __perf_session__process_events (session=0x555555d72fe0)
at util/session.c:2225
#19 0x0000555555861b9d in perf_session__process_events (session=0x555555d72fe0)
at util/session.c:2258
#20 0x0000555555774d02 in __cmd_script (script=0x7fffffffcf30) at builtin-script.c:2557
#21 0x0000555555779988 in cmd_script (argc=0, argv=0x7fffffffebd0)
at builtin-script.c:3926
#22 0x00005555557f2a93 in run_builtin (p=0x555555bb44d8 <commands+408>, argc=4,
argv=0x7fffffffebd0) at perf.c:312
#23 0x00005555557f2d18 in handle_internal_command (argc=4, argv=0x7fffffffebd0)
at perf.c:364
#24 0x00005555557f2e6b in run_argv (argcp=0x7fffffffea2c, argv=0x7fffffffea20)
at perf.c:408#25 0x00005555557f326e in main (argc=4, argv=0x7fffffffebd0) at perf.c:538
This is on today's acme's perf/urgent branch.
Thanks,
Kim
PATCH 1/1 perf script: fix invalid read
closedir(lang_dir) frees the memory of script_dirent->d_name, which gets accessed in the next line in a call to scnprintf().
Valgrind report:
Invalid read of size 1
==413557== at 0x483CBE6: strlen (vg_replace_strmem.c:461) ==413557== by 0x4DD45FD: __vfprintf_internal (vfprintf-internal.c:1688) ==413557== by 0x4DE6679: __vsnprintf_internal (vsnprintf.c:114) ==413557== by 0x53A037: vsnprintf (stdio2.h:80) ==413557== by 0x53A037: scnprintf (vsprintf.c:21) ==413557== by 0x435202: get_script_path (builtin-script.c:3223) ==413557== Address 0x52e7313 is 1,139 bytes inside a block of size 32,816 free'd ==413557== at 0x483AA0C: free (vg_replace_malloc.c:540) ==413557== by 0x4E303C0: closedir (closedir.c:50) ==413557== by 0x4351DC: get_script_path (builtin-script.c:3222)
--- tools/perf/builtin-script.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/builtin-script.c +1 -1
@@ -3218,10 +3218,10 @@ static char *get_script_path(const char *script_root, const char *suffix) __script_root =get_script_root (script_dirent, suffix);if (__script_root && !strcmp (script_root, __script_root)) {free (__script_root); -closedir (lang_dir);closedir (scripts_dir);scnprintf (script_path,MAXPATHLEN ,"%s/%s" , lang_path, script_dirent->d_name); +closedir (lang_dir);return strdup (script_path); }free (__script_root);
Signature · 1 line
2.25.1
Arnaldo Carvalho de Melo
AppliedRe: [PATCH] perf script: fix invalid read
Em Thu, Apr 02, 2020 at 02:43:38PM +0200, Andreas Gerstmayr escreveu:
Quoted 6 lines
closedir(lang_dir)frees the memory ofscript_dirent->d_name, which gets accessed in the next line in a call toscnprintf().Valgrind report:
Invalid read of size 1
Quoted 10 lines
==413557== at 0x483CBE6: strlen (vg_replace_strmem.c:461) ==413557== by 0x4DD45FD: __vfprintf_internal (vfprintf-internal.c:1688) ==413557== by 0x4DE6679: __vsnprintf_internal (vsnprintf.c:114) ==413557== by 0x53A037: vsnprintf (stdio2.h:80) ==413557== by 0x53A037: scnprintf (vsprintf.c:21) ==413557== by 0x435202: get_script_path (builtin-script.c:3223) ==413557== Address 0x52e7313 is 1,139 bytes inside a block of size 32,816 free'd ==413557== at 0x483AA0C: free (vg_replace_malloc.c:540) ==413557== by 0x4E303C0: closedir (closedir.c:50) ==413557== by 0x4351DC: get_script_path (builtin-script.c:3222)
Thanks, applied to perf/urgent.
- Arnaldo
Quoted 24 lines
Signed-off-by Andreas Gerstmayr <[email protected]>--- tools/perf/builtin-script.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/perf/builtin-script.c +1
-1@@ -3218,10 +3218,10 @@ static char *get_script_path(const char *script_root, const char *suffix) __script_root =get_script_root (script_dirent, suffix);if (__script_root && !strcmp (script_root, __script_root)) {free (__script_root); -closedir (lang_dir);closedir (scripts_dir);scnprintf (script_path,MAXPATHLEN ,"%s/%s" , lang_path, script_dirent->d_name); +closedir (lang_dir);return strdup (script_path); }free (__script_root);Signature · 1 line
2.25.1
Signature · 1 line
- Arnaldo
[tip: perf/urgent] perf script: Fix invalid read of directory entry after closedir()
In reply to Andreas GerstmayrThe following commit has been merged into the perf/urgent branch of tip:
The following commit has been merged into the perf/urgent branch of tip:
Commit-ID: 27486a85cb65bd258a9a213b3e95bdf8c24fd781 Gitweb: https://git.kernel.org/tip/27486a85cb65bd258a9a213b3e95bdf8c24fd781 Author: Andreas Gerstmayr <[email protected]> AuthorDate: Thu, 02 Apr 2020 14:43:38 +02:00 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitterDate: Fri, 03 Apr 2020 10:03:18 -03:00
perf script: Fix invalid read of directory entry after closedir()
closedir(lang_dir) frees the memory of script_dirent->d_name, which gets accessed in the next line in a call to scnprintf().
Valgrind report:
Invalid read of size 1
==413557== at 0x483CBE6: strlen (vg_replace_strmem.c:461) ==413557== by 0x4DD45FD: __vfprintf_internal (vfprintf-internal.c:1688) ==413557== by 0x4DE6679: __vsnprintf_internal (vsnprintf.c:114) ==413557== by 0x53A037: vsnprintf (stdio2.h:80) ==413557== by 0x53A037: scnprintf (vsprintf.c:21) ==413557== by 0x435202: get_script_path (builtin-script.c:3223) ==413557== Address 0x52e7313 is 1,139 bytes inside a block of size 32,816 free'd ==413557== at 0x483AA0C: free (vg_replace_malloc.c:540) ==413557== by 0x4E303C0: closedir (closedir.c:50) ==413557== by 0x4351DC: get_script_path (builtin-script.c:3222)
--- tools/perf/builtin-script.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/builtin-script.c +1 -1
@@ -3265,10 +3265,10 @@ static char *get_script_path(const char *script_root, const char *suffix) __script_root =get_script_root (script_dirent, suffix);if (__script_root && !strcmp (script_root, __script_root)) {free (__script_root); -closedir (lang_dir);closedir (scripts_dir);scnprintf (script_path,MAXPATHLEN ,"%s/%s" , lang_path, script_dirent->d_name); +closedir (lang_dir);return strdup (script_path); }free (__script_root);
PATCH 1/1 perf script report: fix segfault when using DWARF mode
In reply to Kim PhillipsWhen running perf script report with a Python script and a callgraph in DWARF mode, intr_regs->regs can be 0 and therefore crashing the regs_map function.
Added a check for this condition (same check as in builtin-script.c:595).
--- tools/perf/util/scripting-engines/trace-event-python.c | 3 +++ 1 file changed, 3 insertions(+)
tools/perf/util/scripting-engines/trace-event-python.c +3 -0
@@ -694,6 +694,9 @@ static int regs_map(struct regs_dump *regs, uint64_t mask, char *bf, int size) bf[0 ] =0 ; +if (!regs || !regs->regs) +return 0 ; +for_each_set_bit (r, (unsigned long *) &mask,sizeof (mask) *8 ) {u64 val = regs->regs[i++];
Signature · 1 line
2.25.1
Arnaldo Carvalho de Melo
AppliedRe: [PATCH] perf script report: fix segfault when using DWARF mode
Em Thu, Apr 02, 2020 at 02:54:16PM +0200, Andreas Gerstmayr escreveu:
Quoted 5 lines
When running perf script report with a Python script and a callgraph in DWARF mode,
intr_regs->regscan be 0 and therefore crashing theregs_mapfunction.Added a check for this condition (same check as in builtin-script.c:595).
Thanks, applied to perf/urgent.
- Arnaldo
Quoted 22 lines
Signed-off-by Andreas Gerstmayr <[email protected]>--- tools/perf/util/scripting-engines/trace-event-python.c | 3 +++ 1 file changed, 3 insertions(+)tools/perf/util/scripting-engines/trace-event-python.c +3
-0@@ -694,6 +694,9 @@ static int regs_map(struct regs_dump *regs, uint64_t mask, char *bf, int size) bf[0 ] =0 ; +if (!regs || !regs->regs) +return 0 ; +for_each_set_bit (r, (unsigned long *) &mask,sizeof (mask) *8 ) {u64 val = regs->regs[i++];Signature · 1 line
2.25.1
Signature · 1 line
- Arnaldo
Re: [PATCH] perf script report: fix segfault when using DWARF mode
In reply to Andreas GerstmayrQuoted 8 lines · Andreas Gerstmayr
When running perf script report with a Python script and a callgraph in DWARF mode,
intr_regs->regscan be 0 and therefore crashing theregs_mapfunction.Added a check for this condition (same check as in builtin-script.c:595).
Signed-off-by Andreas Gerstmayr <[email protected]>---
Thanks,
Kim
Re: [PATCH] perf script report: fix segfault when using DWARF mode
Em Thu, Apr 02, 2020 at 02:07:51PM -0500, Kim Phillips escreveu:
Quoted 11 lines · Andreas Gerstmayr
Quoted 8 lines
When running perf script report with a Python script and a callgraph in DWARF mode,
intr_regs->regscan be 0 and therefore crashing theregs_mapfunction.Added a check for this condition (same check as in builtin-script.c:595).
Signed-off-by Andreas Gerstmayr <[email protected]>---
Tested-by Kim Phillips <[email protected]>
Thanks, added this to that patch.
- Arnaldo
Re: [PATCH] perf script report: fix segfault when using DWARF mode
Quoted 6 lines · Arnaldo Carvalho de Melo
Em Thu, Apr 02, 2020 at 02:07:51PM -0500, Kim Phillips escreveu:
Quoted 5 lines · Andreas Gerstmayr
Quoted 4 lines
When running perf script report with a Python script and a callgraph in DWARF mode,
intr_regs->regscan be 0 and therefore crashing theregs_mapfunction.
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
Added a check for this condition (same check as in builtin-script.c:595).
Signed-off-by Andreas Gerstmayr <[email protected]>---
Tested-by Kim Phillips <[email protected]>Thanks, added this to that patch.
Great, thanks!
Cheers,
Andreas
Re: [PATCH] perf script report: fix segfault when using DWARF mode
Quoted 12 lines · Andreas Gerstmayr
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Thu, Apr 02, 2020 at 02:07:51PM -0500, Kim Phillips escreveu:
Quoted 10 lines · Andreas Gerstmayr
Quoted 8 lines
When running perf script report with a Python script and a callgraph in DWARF mode,
intr_regs->regscan be 0 and therefore crashing theregs_mapfunction.Added a check for this condition (same check as in builtin-script.c:595).
Signed-off-by Andreas Gerstmayr <[email protected]>---
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Tested-by Kim Phillips <[email protected]>Thanks, added this to that patch.
Great, thanks!
Ah, I thought you were referring to my initial flamegraph.py perf script. Is there anything I can do to get it merged?
Cheers,
Andreas
Re: [PATCH] perf script report: fix segfault when using DWARF mode
Em Mon, Apr 06, 2020 at 11:30:23AM +0200, Andreas Gerstmayr escreveu:
Quoted 14 lines · Andreas Gerstmayr
Quoted 12 lines · Arnaldo Carvalho de Melo
Quoted 11 lines
Em Thu, Apr 02, 2020 at 02:07:51PM -0500, Kim Phillips escreveu:
Quoted 10 lines · Andreas Gerstmayr
Quoted 9 lines
When running perf script report with a Python script and a callgraph in DWARF mode,
intr_regs->regscan be 0 and therefore crashing theregs_map
function.Added a check for this condition (same check as in builtin-script.c:595).
Signed-off-by Andreas Gerstmayr <[email protected]>
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Tested-by Kim Phillips <[email protected]>Thanks, added this to that patch.
Great, thanks!
Ah, I thought you were referring to my initial
flamegraph.pyperf script. Is there anything I can do to get it merged?
I'll test it today, were there any Tested-by: or Reviewed-by: to that flamegraph.py?
That is not yet a strict requirement for having patches accepted, but help me a lot in dedicating time to test things that passed thru some testing by other people,
Thanks,
- Arnaldo
Re: [PATCH] perf script report: fix segfault when using DWARF mode
Quoted 12 lines · Arnaldo Carvalho de Melo
Em Mon, Apr 06, 2020 at 11:30:23AM +0200, Andreas Gerstmayr escreveu:
Quoted 11 lines · Andreas Gerstmayr
Quoted 10 lines · Arnaldo Carvalho de Melo
Quoted 9 lines
Em Thu, Apr 02, 2020 at 02:07:51PM -0500, Kim Phillips escreveu:
Quoted 8 lines · Andreas Gerstmayr
Quoted 7 lines
When running perf script report with a Python script and a callgraph in DWARF mode,
intr_regs->regscan be 0 and therefore crashing theregs_map
function.
Added a check for this condition (same check as in builtin-script.c:595).Signed-off-by Andreas Gerstmayr <[email protected]>
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 2 lines
Quoted 1 line
Tested-by Kim Phillips <[email protected]>Thanks, added this to that patch.
Great, thanks!
Ah, I thought you were referring to my initial
flamegraph.pyperf script. Is there anything I can do to get it merged?I'll test it today, were there any Tested-by: or Reviewed-by: to that
flamegraph.py?
I just added mine.
Kim
[tip: perf/urgent] perf script report: Fix SEGFAULT when using DWARF mode
In reply to Andreas GerstmayrThe following commit has been merged into the perf/urgent branch of tip:
The following commit has been merged into the perf/urgent branch of tip:
Commit-ID: 1a4025f06059eeaecb2ef24363350ea3431568df Gitweb: https://git.kernel.org/tip/1a4025f06059eeaecb2ef24363350ea3431568df Author: Andreas Gerstmayr <[email protected]> AuthorDate: Thu, 02 Apr 2020 14:54:16 +02:00 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitterDate: Fri, 03 Apr 2020 09:39:53 -03:00
perf script report: Fix SEGFAULT when using DWARF mode
When running perf script report with a Python script and a callgraph in DWARF mode, intr_regs->regs can be 0 and therefore crashing the regs_map function.
Added a check for this condition (same check as in builtin-script.c:595).
--- tools/perf/util/scripting-engines/trace-event-python.c | 3 +++ 1 file changed, 3 insertions(+)
tools/perf/util/scripting-engines/trace-event-python.c +3 -0
@@ -694,6 +694,9 @@ static int regs_map(struct regs_dump *regs, uint64_t mask, char *bf, int size) bf[0 ] =0 ; +if (!regs || !regs->regs) +return 0 ; +for_each_set_bit (r, (unsigned long *) &mask,sizeof (mask) *8 ) {u64 val = regs->regs[i++];
Quoted 8 lines · Kim Phillips
I now get a SIGSEGV when executing perf script report flamegraph.
Here's a trace:
#0
0x000055555590a9b2inregs_map(regs=0x7fffffffbfc8, mask=16715775,bf=0x7fffffffba60 "", size=512) at util/scripting-engines/trace-event-python.c:696 #1 0x000055555590ab03 in set_regs_in_dict (dict=0x7ffff61dd500, sample=0x7fffffffbf20, evsel=0x555555d7a700) at util/scripting-engines/trace-event-python.c:718
This error seems unrelated to flamegraph.py (occurs also with the stackcollapse.py script).
Looks like the intr_regs->regs can be 0 when running in DWARF mode (this error doesn't occur in the default mode). I've added a check and sent a patch. While at it, valgrind reported an invalid read in a different code path, which is fixed by the other patch I just sent a few minutes ago.
Can you please test again and report if there are any other issues?
Cheers,
Andreas
Kim Phillips
Tested-byQuoted 10 lines · Andreas Gerstmayr
Quoted 8 lines · Kim Phillips
I now get a SIGSEGV when executing perf script report flamegraph.
Here's a trace:
#0
0x000055555590a9b2inregs_map(regs=0x7fffffffbfc8, mask=16715775, bf=0x7fffffffba60 "", size=512) atutil/scripting-engines/trace-event-python.c:696#10x000055555590ab03inset_regs_in_dict(dict=0x7ffff61dd500, sample=0x7fffffffbf20, evsel=0x555555d7a700) atutil/scripting-engines/trace-event-python.c:718
Quoted 5 lines
This error seems unrelated to
flamegraph.py(occurs also with thestackcollapse.pyscript).Looks like the
intr_regs->regscan be 0 when running in DWARF mode (this error doesn't occur in the default mode). I've added a check and sent a patch. While at it, valgrind reported an invalid read in a different code path, which is fixed by the other patch I just sent a few minutes ago.Can you please test again and report if there are any other issues?
This flamegraph patch works for me now:
Thanks,
Kim
Em Fri, Mar 20, 2020 at 04:13:48PM +0100, Andreas Gerstmayr escreveu:
Quoted 8 lines
This script works in tandem with d3-flame-graph to generate flame graphs from perf. It supports two output formats: JSON and HTML (the default). The HTML format will look for a standalone d3-flame-graph template file in
/usr/share/d3-flame-graph/d3-flamegraph-base.htmland fill in the collected stacks.Usage:
Quoted 8 lines
perf record -a -g -F 99 sleep 60
perf script report flamegraphCombined:
perf script flamegraph -a -F 99 sleep 60
Signed-off-by Andreas Gerstmayr <[email protected]>
Trying it now with:
[root@five ~]# perf record -a -g -F 99 sleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 1.120 MB perf.data (59 samples) ] [root@five ~]# perf script report flamegraph
File "/home/acme/libexec/perf-core/scripts/python/flamegraph.py", line 46
print(f"Flame Graph template {self.args.template} does not " +
^
SyntaxError: invalid syntax
Error running python script /home/acme/libexec/perf-core/scripts/python/flamegraph.py [root@five ~]#
Ok, gets better when I build with 'make -C tools/perf PYTHON=python3'
[root@five ~]# perf script report flamegraph
Flame Graph template /usr/share/d3-flame-graph/d3-flamegraph-base.html does not exist. Please install the js-d3-flame-graph (RPM) or libjs-d3-flame-graph (deb) package, specify an existing flame graph template (--template PATH) or another output format (--format FORMAT). [root@five ~]#
It works then, so I'll add a note to that effect. But please consider fixing it so that it works with both python2 and python3, if possible, or to fail gracefully and state that only python3 is supported and that perf should be built with "PYTHON=python3" on the make command line.
Some RFEs for you to consider:
1. make:
perf flamegraph -a -F 99 sleep 1
be equivalent, should be easy.
2. make the command somehow create a new tab on an existing browser with the generated file, and switch the focus to that tab.
3. get whats in:
[root@five ~]# perf report --header-only
# ========
# captured on : Mon Apr 6 12:02:52 2020
# header version : 1
# data offset : 336
# data size : 1173584
# feat offset : 1173920
# hostname : five
# os release : 5.5.10-200.fc31.x86_64
# perf version : 5.6.gb6b7dc6ec0c8
# arch : x86_64
# nrcpus online : 12
# nrcpus avail : 12
# cpudesc : AMD Ryzen 5 3600X 6-Core Processor
# cpuid : AuthenticAMD,23,113,0
# total memory : 32890748 kB
# cmdline : /home/acme/bin/perf record -a -g -F 99 sleep 1
# event : name = cycles, , id = { 2918625, 2918626, 2918627, 2918628, 2918629, 2918630, 2918631, 2918632, 2918633, 2918634, 2918635, 2918636 }, size = 120, { sample_period, sample_freq } = 99, sample_type = IP|TID|TIME|CALLCHAIN|CPU|PERIOD, read_format = ID, disabled = 1,>
# CPU_TOPOLOGY info available, use -I to display
# NUMA_TOPOLOGY info available, use -I to display
# pmu mappings: amd_df = 8, software = 1, ibs_op = 11, ibs_fetch = 10, uprobe = 7, cpu = 4, amd_iommu_0 = 12, breakpoint = 5, amd_l3 = 9, tracepoint = 2, kprobe = 6, msr = 13
# CACHE info available, use -I to display
# time of first sample : 87600.831767
# time of last sample : 87601.096829
# sample duration : 265.062 ms
# MEM_TOPOLOGY info available, use -I to display
# bpf_prog_info 40: bpf_prog_6deef7357e7b4530 addr 0xffffffffc030650c size 66
# bpf_prog_info 41: bpf_prog_6deef7357e7b4530 addr 0xffffffffc03080e8 size 66
# bpf_prog_info 42: bpf_prog_6deef7357e7b4530 addr 0xffffffffc030a218 size 66
# bpf_prog_info 43: bpf_prog_6deef7357e7b4530 addr 0xffffffffc036c698 size 66
# bpf_prog_info 44: bpf_prog_5a2b06eab81b8f51 addr 0xffffffffc036e2c0 size 1132
# bpf_prog_info 45: bpf_prog_6deef7357e7b4530 addr 0xffffffffc06e7118 size 66
# bpf_prog_info 46: bpf_prog_6deef7357e7b4530 addr 0xffffffffc06e9cd8 size 66
# bpf_prog_info 47: bpf_prog_f3b9e166f6c1aaaa addr 0xffffffffc089732c size 1786
# bpf_prog_info 48: bpf_prog_6deef7357e7b4530 addr 0xffffffffc0dcb64c size 66
# bpf_prog_info 49: bpf_prog_6deef7357e7b4530 addr 0xffffffffc0dcd0fc size 66
# bpf_prog_info 50: bpf_prog_6deef7357e7b4530 addr 0xffffffffc0f4d8dc size 66
# bpf_prog_info 51: bpf_prog_6deef7357e7b4530 addr 0xffffffffc0f4fc7c size 66
# bpf_prog_info 52: bpf_prog_84efc2eecc454ca6 addr 0xffffffffc0f6e584 size 373
# missing features: TRACING_DATA BRANCH_STACK GROUP_DESC AUXTRACE STAT CLOCKID DIR_FORMAT COMPRESSED
# ========
#
[root@five ~]#And make it available in some UI element.
Thanks,
- Arnaldo
PATCH 1/1 perf script flamegraph: python2 support, update cli args
* removed --indent argument
* renamed -F to -f argument to be consistent with other arguments
--- tools/perf/scripts/python/flamegraph.py | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)
tools/perf/scripts/python/flamegraph.py +11 -13
@@ -14,6 +14,7 @@ # Flame Graphs invented by Brendan Gregg <[email protected]> # Works in tandem with d3-flame-graph by Martin Spier <[email protected]> +from __future__ import print_function import sys import os import argparse
@@ -43,11 +44,11 @@ class FlameGraphCLI: if self.args.format == "html" and \ not os.path.isfile(self.args.template): - print(f"Flame Graph template {self.args.template} does not " + - f"exist. Please install the js-d3-flame-graph (RPM) or " + - f"libjs-d3-flame-graph (deb) package, specify an " + - f"existing flame graph template (--template PATH) or " + - f"another output format (--format FORMAT).", + print("Flame Graph template {} does not exist. Please install " + "the js-d3-flame-graph (RPM) or libjs-d3-flame-graph (deb) " + "package, specify an existing flame graph template " + "(--template PATH) or another output format " + "(--format FORMAT).".format(self.args.template), file=sys.stderr) sys.exit(1)
@@ -76,8 +77,7 @@ class FlameGraphCLI: node.value += 1 def trace_end(self): - json_str = json.dumps(self.stack, default=lambda x: x.toJSON(), - indent=self.args.indent) + json_str = json.dumps(self.stack, default=lambda x: x.toJSON()) if self.args.format == "html": try:
@@ -85,7 +85,7 @@ class FlameGraphCLI: output_str = f.read().replace("/** @flamegraph_json **/", json_str) except IOError as e: - print(f"Error reading template file: {e}", file=sys.stderr) + print("Error reading template file: {}".format(e), file=sys.stderr) sys.exit(1) output_fn = self.args.output or "flamegraph.html" else:
@@ -95,24 +95,22 @@ class FlameGraphCLI: if output_fn == "-": sys.stdout.write(output_str) else: - print(f"dumping data to {output_fn}") + print("dumping data to {}".format(output_fn)) try: with open(output_fn, "w") as out: out.write(output_str) except IOError as e: - print(f"Error writing output file: {e}", file=sys.stderr) + print("Error writing output file: {}".format(e), file=sys.stderr) sys.exit(1) if __name__ == "__main__": parser = argparse.ArgumentParser(description="Create flame graphs.") - parser.add_argument("-F", "--format", + parser.add_argument("-f", "--format", default="html", choices=["json", "html"], help="output file format") parser.add_argument("-o", "--output", help="output file name") - parser.add_argument("--indent", - type=int, help="JSON indentation") parser.add_argument("--template", default="/usr/share/d3-flame-graph/d3-flamegraph-base.html", help="path to flamegraph HTML template")
Signature · 1 line
2.25.2
Re: [PATCH] perf script flamegraph: python2 support, update cli args
Em Thu, Apr 09, 2020 at 06:57:31PM +0200, Andreas Gerstmayr escreveu:
Quoted 2 lines
* removed --indent argument
* renamed -F to -f argument to be consistent with other arguments
Ok, since I hadn't push the original patch, I've combined this one with it it so that in systems with python2 it starts working and we can bisect things from a good starting point.
- Arnaldo
Quoted 84 lines
Signed-off-by Andreas Gerstmayr <[email protected]>--- tools/perf/scripts/python/flamegraph.py | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)tools/perf/scripts/python/flamegraph.py +11
-13@@ -14,6 +14,7 @@ # Flame Graphs invented by Brendan Gregg <[email protected]> # Works in tandem with d3-flame-graph by Martin Spier <[email protected]> +from __future__ import print_function import sys import os import argparse@@ -43,11 +44,11 @@ class FlameGraphCLI: if self.args.format == "html" and \ not os.path.isfile(self.args.template): - print(f"Flame Graph template {self.args.template} does not " + - f"exist. Please install the js-d3-flame-graph (RPM) or " + - f"libjs-d3-flame-graph (deb) package, specify an " + - f"existing flame graph template (--template PATH) or " + - f"another output format (--format FORMAT).", + print("Flame Graph template {} does not exist. Please install " + "the js-d3-flame-graph (RPM) or libjs-d3-flame-graph (deb) " + "package, specify an existing flame graph template " + "(--template PATH) or another output format " + "(--format FORMAT).".format(self.args.template), file=sys.stderr) sys.exit(1)@@ -76,8 +77,7 @@ class FlameGraphCLI: node.value += 1 def trace_end(self): - json_str = json.dumps(self.stack, default=lambda x: x.toJSON(), - indent=self.args.indent) + json_str = json.dumps(self.stack, default=lambda x: x.toJSON()) if self.args.format == "html": try:@@ -85,7 +85,7 @@ class FlameGraphCLI: output_str = f.read().replace("/** @flamegraph_json **/", json_str) except IOError as e: - print(f"Error reading template file: {e}", file=sys.stderr) + print("Error reading template file: {}".format(e), file=sys.stderr) sys.exit(1) output_fn = self.args.output or "flamegraph.html" else:@@ -95,24 +95,22 @@ class FlameGraphCLI: if output_fn == "-": sys.stdout.write(output_str) else: - print(f"dumping data to {output_fn}") + print("dumping data to {}".format(output_fn)) try: with open(output_fn, "w") as out: out.write(output_str) except IOError as e: - print(f"Error writing output file: {e}", file=sys.stderr) + print("Error writing output file: {}".format(e), file=sys.stderr) sys.exit(1) if __name__ == "__main__": parser = argparse.ArgumentParser(description="Create flame graphs.") - parser.add_argument("-F", "--format", + parser.add_argument("-f", "--format", default="html", choices=["json", "html"], help="output file format") parser.add_argument("-o", "--output", help="output file name") - parser.add_argument("--indent", - type=int, help="JSON indentation") parser.add_argument("--template", default="/usr/share/d3-flame-graph/d3-flamegraph-base.html", help="path to flamegraph HTML template")Signature · 1 line
2.25.2
Signature · 1 line
- Arnaldo
Hi Arnaldo,
thanks for merging!
Quoted 4 lines · Arnaldo Carvalho de Melo
It works then, so I'll add a note to that effect. But please consider fixing it so that it works with both python2 and python3, if possible, or to fail gracefully and state that only python3 is supported and that perf should be built with "PYTHON=python3" on the make command line.
Ok, I just sent a patch (based on your perf/core branch) to add Python 2 support. I hope soon Python 2 support won't be required anymore ;)
Quoted 7 lines
Some RFEs for you to consider:
1. make:
perf flamegraph -a -F 99 sleep 1
be equivalent, should be easy.
perf flamegraph -a -F 99 sleep 1 should be equivalent to perf script flamegraph -a -F 99 sleep 1? Or in other words, flamegraph should be a top-level command of perf, and run the flamegraph.py script?
Quoted 2 lines
2. make the command somehow create a new tab on an existing browser with the generated file, and switch the focus to that tab.
As a personal preference I don't like programs interfering with my browser session. For example I have a flame graph open in a tab, then I generate another one (overwriting the current file), I would not want another tab to open with the new flame graph, but instead I want to manually refresh the currently open tab.
Quoted 46 lines
3. get whats in:
[root@five ~]# perf report --header-only
# ========# captured on : Mon Apr 6 12:02:52 2020 # header version : 1 # data offset : 336 # data size : 1173584 # feat offset : 1173920 # hostname : five # os release : 5.5.10-200.fc31.x86_64 # perf version : 5.6.gb6b7dc6ec0c8 # arch : x86_64 # nrcpus online : 12 # nrcpus avail : 12 # cpudesc : AMD Ryzen 5 3600X 6-Core Processor # cpuid : AuthenticAMD,23,113,0 # total memory : 32890748 kB # cmdline : /home/acme/bin/perf record -a -g -F 99 sleep 1 # event : name = cycles, , id = { 2918625, 2918626, 2918627, 2918628, 2918629, 2918630, 2918631, 2918632, 2918633, 2918634, 2918635, 2918636 }, size = 120, { sample_period, sample_freq } = 99, sample_type = IP|TID|TIME|CALLCHAIN|CPU|PERIOD, read_format = ID, disabled = 1,> # CPU_TOPOLOGY info available, use -I to display # NUMA_TOPOLOGY info available, use -I to display # pmu mappings: amd_df = 8, software = 1, ibs_op = 11, ibs_fetch = 10, uprobe = 7, cpu = 4, amd_iommu_0 = 12, breakpoint = 5, amd_l3 = 9, tracepoint = 2, kprobe = 6, msr = 13 # CACHE info available, use -I to display # time of first sample : 87600.831767 # time of last sample : 87601.096829 # sample duration : 265.062 ms # MEM_TOPOLOGY info available, use -I to display # bpf_prog_info 40: bpf_prog_6deef7357e7b4530 addr 0xffffffffc030650c size 66 # bpf_prog_info 41: bpf_prog_6deef7357e7b4530 addr 0xffffffffc03080e8 size 66 # bpf_prog_info 42: bpf_prog_6deef7357e7b4530 addr 0xffffffffc030a218 size 66 # bpf_prog_info 43: bpf_prog_6deef7357e7b4530 addr 0xffffffffc036c698 size 66 # bpf_prog_info 44: bpf_prog_5a2b06eab81b8f51 addr 0xffffffffc036e2c0 size 1132 # bpf_prog_info 45: bpf_prog_6deef7357e7b4530 addr 0xffffffffc06e7118 size 66 # bpf_prog_info 46: bpf_prog_6deef7357e7b4530 addr 0xffffffffc06e9cd8 size 66 # bpf_prog_info 47: bpf_prog_f3b9e166f6c1aaaa addr 0xffffffffc089732c size 1786 # bpf_prog_info 48: bpf_prog_6deef7357e7b4530 addr 0xffffffffc0dcb64c size 66 # bpf_prog_info 49: bpf_prog_6deef7357e7b4530 addr 0xffffffffc0dcd0fc size 66 # bpf_prog_info 50: bpf_prog_6deef7357e7b4530 addr 0xffffffffc0f4d8dc size 66 # bpf_prog_info 51: bpf_prog_6deef7357e7b4530 addr 0xffffffffc0f4fc7c size 66 # bpf_prog_info 52: bpf_prog_84efc2eecc454ca6 addr 0xffffffffc0f6e584 size 373 # missing features: TRACING_DATA BRANCH_STACK GROUP_DESC AUXTRACE STAT CLOCKID DIR_FORMAT COMPRESSED # ======== # [root@five ~]#
Quoted 1 line
And make it available in some UI element.
Good idea!
I'll implement this with the next set of flame graph template changes.
Thanks,
Andreas
[tip: perf/core] perf script: Add flamegraph.py script
In reply to Andreas GerstmayrThe following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 5287f926920688e1151741d49da37a533ccf1960 Gitweb: https://git.kernel.org/tip/5287f926920688e1151741d49da37a533ccf1960 Author: Andreas Gerstmayr <[email protected]> AuthorDate: Fri, 20 Mar 2020 16:13:48 +01:00 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitterDate: Thu, 16 Apr 2020 12:19:14 -03:00
perf script: Add flamegraph.py script
This script works in tandem with d3-flame-graph to generate flame graphs from perf. It supports two output formats: JSON and HTML (the default). The HTML format will look for a standalone d3-flame-graph template file in /usr/share/d3-flame-graph/d3-flamegraph-base.html and fill in the collected stacks.
Usage:
perf record -a -g -F 99 sleep 60
perf script report flamegraph
Combined:
perf script flamegraph -a -F 99 sleep 60
Committer testing:
Tested both with "PYTHON=python3" and with the default, that uses python2-devel:
Complete set of instructions:
$ mkdir /tmp/build/perf $ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin $ export PATH=~/bin:$PATH $ perf record -a -g -F 99 sleep 60 $ perf script report flamegraph
Now go and open the generated flamegraph.html file in a browser.
At first this required building with PYTHON=python3, but after I reported this Andreas was kind enough to send a patch making it work with both python and python3.
--- tools/perf/scripts/python/bin/flamegraph-record | 2 +- tools/perf/scripts/python/bin/flamegraph-report | 3 +- tools/perf/scripts/python/flamegraph.py | 124 +++++++++++++++- 3 files changed, 129 insertions(+) create mode 100755 tools/perf/scripts/python/bin/flamegraph-record create mode 100755 tools/perf/scripts/python/bin/flamegraph-report create mode 100755 tools/perf/scripts/python/flamegraph.py
tools/perf/scripts/python/bin/flamegraph-record new file +2 -0
@@ -0,0 +1,2 @@ +#!/usr/bin/sh +perf record -g "$@"
tools/perf/scripts/python/bin/flamegraph-report new file +3 -0
@@ -0,0 +1,3 @@ +#!/usr/bin/sh +# description: create flame graphs +perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
tools/perf/scripts/python/flamegraph.py new file +124 -0
124 lines not on this page · raw message