Skip to contentSkip to conversation

perf script: how to print user-space backtrace?

PATCH In review Milian Wolff 5 participants 6 messages Jul 22, 2015 – Aug 20, 2015 perf events

lore
Share & tools
lore
#0 lore raw

Hey all,

I recorded a perf data file using

perf record --call-graph dwarf kwrite <some large file>

when I report it I see backtraces pointing to the user space code:

perf report --stdio -g graph --no-children

    16.36%  kwrite           libKF5TextEditor.so.5.13.0     [.] 
QArrayData::data                                                                                               
                     |
                     ---QArrayData::data
                        |          
                        |--15.46%-- QTypedArrayData<unsigned short>::data

but when I run perf script, I do not see _any_ backtraces into user space.

perf script -F comm,tid,time,event,ip,sym,trace,period
:4898 4898 17486.943365: 1 cycles:

                  25f83a native_write_msr_safe
                  234f8a __intel_pmu_enable_all
                  235010 intel_pmu_enable_all
                  22e1ec x86_pmu_enable
                  356fb7 perf_pmu_enable.part.53
                  35824d perf_event_context_sched_in.isra.64
                  358e0b perf_event_exec
                  3e8c93 setup_new_exec
                  439a9f load_elf_binary
                  3e7f48 search_binary_handler
                  3e89cf do_execveat_common.isra.15
                  3e8e4a sys_execve
                  78b895 return_from_execve
kwrite  4898 17486.943893:          1 cycles: 
kwrite  4898 17486.944211:          1 cycles: 
...
kwrite  4898 17486.953664:     461703 cycles: 
kwrite  4898 17486.953838:     490150 cycles: 
                  78d4c0 page_fault
kwrite  4898 17486.954009:     516694 cycles: 
...
kwrite  4898 17510.765568:     766937 cycles: 
                  78979b down_write
                  39bd82 unlink_file_vma
                  3944d0 free_pgtables
                  39eef6 exit_mmap
                  275d45 mmput
                  27b514 do_exit
                  27bdfb do_group_exit
                  27be84 [unknown]
                  78b56e system_call

So it looks to me as if only kernel-space call stacks are unwound in perf script. Is there a magic switch that I'm missing to get user-space stacks unwound? Note I'm using perf 4.1.0 on a Linux 4.1.2 system.

Thanks

Signature · 3 lines
Milian Wolff
[email protected]
http://milianw.de
#1 lore raw

PATCH 1/1 perf script: Initialize callchain_param.record_mode

Quoted 53 lines · Milian Wolff

Hey all,

I recorded a perf data file using

perf record --call-graph dwarf kwrite <some large file>

when I report it I see backtraces pointing to the user space code:

perf report --stdio -g graph --no-children

    16.36%  kwrite           libKF5TextEditor.so.5.13.0     [.] 
QArrayData::data                                                                                               
                     |
                     ---QArrayData::data
                        |          
                        |--15.46%-- QTypedArrayData<unsigned short>::data

but when I run perf script, I do not see _any_ backtraces into user space.

perf script -F comm,tid,time,event,ip,sym,trace,period
:4898 4898 17486.943365: 1 cycles:

                  25f83a native_write_msr_safe
                  234f8a __intel_pmu_enable_all
                  235010 intel_pmu_enable_all
                  22e1ec x86_pmu_enable
                  356fb7 perf_pmu_enable.part.53
                  35824d perf_event_context_sched_in.isra.64
                  358e0b perf_event_exec
                  3e8c93 setup_new_exec
                  439a9f load_elf_binary
                  3e7f48 search_binary_handler
                  3e89cf do_execveat_common.isra.15
                  3e8e4a sys_execve
                  78b895 return_from_execve
kwrite  4898 17486.943893:          1 cycles: 
kwrite  4898 17486.944211:          1 cycles: 
...
kwrite  4898 17486.953664:     461703 cycles: 
kwrite  4898 17486.953838:     490150 cycles: 
                  78d4c0 page_fault
kwrite  4898 17486.954009:     516694 cycles: 
...
kwrite  4898 17510.765568:     766937 cycles: 
                  78979b down_write
                  39bd82 unlink_file_vma
                  3944d0 free_pgtables
                  39eef6 exit_mmap
                  275d45 mmput
                  27b514 do_exit
                  27bdfb do_group_exit
                  27be84 [unknown]
                  78b56e system_call
Quoted 3 lines

So it looks to me as if only kernel-space call stacks are unwound in perf script. Is there a magic switch that I'm missing to get user-space stacks unwound? Note I'm using perf 4.1.0 on a Linux 4.1.2 system.

hi,
right you are.. does attached patch help?

thanks,
jirka

---
Milian Wolff reported non functional DWARF unwind under perf script. The reason is that perf script does not properly configure callchain_param.record_mode, which is needed by unwind code.

Stealing the code from report and leaving the place for more initialization code in a hope we could merge it with report__setup_sample_type one day.

---
 tools/perf/builtin-script.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tools/perf/builtin-script.c +17 -0
index 7b376d215e94..105332e950a9 100644
@@ -1561,6 +1561,22 @@ static int have_cmd(int argc, const char **argv)
 	return 0;
 }
 
+static void script__setup_sample_type(struct perf_script *script)
+{
+	struct perf_session *session = script->session;
+	u64 sample_type = perf_evlist__combined_sample_type(session->evlist);
+
+	if (symbol_conf.use_callchain || symbol_conf.cumulate_callchain) {
+		if ((sample_type & PERF_SAMPLE_REGS_USER) &&
+		    (sample_type & PERF_SAMPLE_STACK_USER))
+			callchain_param.record_mode = CALLCHAIN_DWARF;
+		else if (sample_type & PERF_SAMPLE_BRANCH_STACK)
+			callchain_param.record_mode = CALLCHAIN_LBR;
+		else
+			callchain_param.record_mode = CALLCHAIN_FP;
+	}
+}
+
 int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
 {
 	bool show_full_info = false;
@@ -1849,6 +1865,7 @@ int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
 		goto out_delete;
 
 	script.session = session;
+	script__setup_sample_type(&script);
 
 	session->itrace_synth_opts = &itrace_synth_opts;
 
Signature · 1 line
2.4.3
#2 lore raw

Re: [PATCH] perf script: Initialize callchain_param.record_mode

Quoted 60 lines · Jiri Olsa
Quoted 59 lines · Milian Wolff

Hey all,

I recorded a perf data file using

perf record --call-graph dwarf kwrite <some large file>

when I report it I see backtraces pointing to the user space code:

perf report --stdio -g graph --no-children

    16.36%  kwrite           libKF5TextEditor.so.5.13.0     [.]

QArrayData::data

                     ---QArrayData::data
                     
                        |--15.46%-- QTypedArrayData<unsigned short>::data

but when I run perf script, I do not see _any_ backtraces into user space.

perf script -F comm,tid,time,event,ip,sym,trace,period

:4898 4898 17486.943365: 1 cycles:

                  25f83a native_write_msr_safe
                  234f8a __intel_pmu_enable_all
                  235010 intel_pmu_enable_all
                  22e1ec x86_pmu_enable
                  356fb7 perf_pmu_enable.part.53
                  35824d perf_event_context_sched_in.isra.64
                  358e0b perf_event_exec
                  3e8c93 setup_new_exec
                  439a9f load_elf_binary
                  3e7f48 search_binary_handler
                  3e89cf do_execveat_common.isra.15
                  3e8e4a sys_execve
                  78b895 return_from_execve
kwrite  4898 17486.943893:          1 cycles:
kwrite  4898 17486.944211:          1 cycles:

...
kwrite 4898 17486.953664: 461703 cycles:

kwrite 4898 17486.953838: 490150 cycles:

                  78d4c0 page_fault

kwrite 4898 17486.954009: 516694 cycles:
...

kwrite 4898 17510.765568: 766937 cycles:

                  78979b down_write
                  39bd82 unlink_file_vma
                  3944d0 free_pgtables
                  39eef6 exit_mmap
                  275d45 mmput
                  27b514 do_exit
                  27bdfb do_group_exit
                  27be84 [unknown]
                  78b56e system_call
Quoted 6 lines
Quoted 3 lines

So it looks to me as if only kernel-space call stacks are unwound in perf script. Is there a magic switch that I'm missing to get user-space stacks unwound? Note I'm using perf 4.1.0 on a Linux 4.1.2 system.

hi,
right you are.. does attached patch help?

<snip>

Hey Jiri,

it works a treat!

Many thanks, much appreciated.

Signature · 4 lines
Milian Wolff | [email protected] | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt Experts
#3 lore raw

Re: [PATCH] perf script: Initialize callchain_param.record_mode

Em Thu, Aug 13, 2015 at 10:53:36AM +0200, Milian Wolff escreveu:

Quoted 10 lines · Jiri Olsa
Quoted 1 line

right you are.. does attached patch help?

<snip>

Hey Jiri,

it works a treat!

Many thanks, much appreciated.

Adding a "Tested-by: Milian", Ok?

- Arnaldo

#4 lore raw

Re: [PATCH] perf script: Initialize callchain_param.record_mode

Quoted 6 lines · CEST Arnaldo Carvalho de Melo

Em Thu, Aug 13, 2015 at 10:53:36AM +0200, Milian Wolff escreveu:

Quoted 5 lines · Jiri Olsa
Quoted 1 line

right you are.. does attached patch help?

<snip>

Quoted 7 lines
Quoted 5 lines

Hey Jiri,

it works a treat!

Many thanks, much appreciated.

Adding a "Tested-by: Milian", Ok?

Sure, please go ahead.

Thanks

Signature · 4 lines
Milian Wolff | [email protected] | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt Experts
#5 lore raw

[tip:perf/core] perf script: Initialize callchain_param.record_mode

In reply to Jiri Olsa
perf script: Initialize callchain_param.record_mode
Commit-ID:  7322d6c98dd214252bd697f8dde64a3576977fab
Gitweb:     http://git.kernel.org/tip/7322d6c98dd214252bd697f8dde64a3576977fab
Author:     Jiri Olsa <[email protected]>
AuthorDate: Thu, 13 Aug 2015 09:17:24 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Mon, 17 Aug 2015 10:48:39 -0300

perf script: Initialize callchain_param.record_mode

Milian Wolff reported non functional DWARF unwind under perf script. The reason is that perf script does not properly configure callchain_param.record_mode, which is needed by unwind code.

Stealing the code from report and leaving the place for more initialization code in a hope we could merge it with report__setup_sample_type one day.

---
 tools/perf/builtin-script.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tools/perf/builtin-script.c +17 -0
index 7b376d2..105332e 100644
@@ -1561,6 +1561,22 @@ static int have_cmd(int argc, const char **argv)
 	return 0;
 }
 
+static void script__setup_sample_type(struct perf_script *script)
+{
+	struct perf_session *session = script->session;
+	u64 sample_type = perf_evlist__combined_sample_type(session->evlist);
+
+	if (symbol_conf.use_callchain || symbol_conf.cumulate_callchain) {
+		if ((sample_type & PERF_SAMPLE_REGS_USER) &&
+		    (sample_type & PERF_SAMPLE_STACK_USER))
+			callchain_param.record_mode = CALLCHAIN_DWARF;
+		else if (sample_type & PERF_SAMPLE_BRANCH_STACK)
+			callchain_param.record_mode = CALLCHAIN_LBR;
+		else
+			callchain_param.record_mode = CALLCHAIN_FP;
+	}
+}
+
 int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
 {
 	bool show_full_info = false;
@@ -1849,6 +1865,7 @@ int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
 		goto out_delete;
 
 	script.session = session;
+	script__setup_sample_type(&script);
 
 	session->itrace_synth_opts = &itrace_synth_opts;