perf kvm: fix segfault with default guest arguments
PATCH 1/1 perf kvm: fix segfault with default guest arguments
(This time with actual backtrace; forgot to remove the leading '#' on the frames.)
File generated with:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms
--guestmodules=/tmp/guest-modules record -aThe report command:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms
--guestmodules=/tmp/guest-modules reportdies with a SEGFAULT:
Program received signal SIGSEGV, Segmentation fault.0x000000000046dd7b in machine__mmap_name (self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) at util/map.c:715
715 snprintf(bf, size, "[%s.%d]", "guest.kernel.kallsyms", self->pid);
(gdb) bt
0 0x000000000046dd7b in machine__mmap_name (self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) at util/map.c:715
1 0x0000000000444161 in perf_event__process_kernel_mmap (tool=0x7fffffffdd80, event=0x7ffff7fb4120, machine=0x0) at util/event.c:562
2 0x0000000000444642 in perf_event__process_mmap (tool=0x7fffffffdd80, event=0x7ffff7fb4120, sample=0x7fffffffd210, machine=0x0)
at util/event.c:668
3 0x0000000000470e0b in perf_session_deliver_event (session=0x915ca0, event=0x7ffff7fb4120, sample=0x7fffffffd210, tool=0x7fffffffdd80,
file_offset=8480) at util/session.c:979
4 0x000000000047032e in flush_sample_queue (s=0x915ca0, tool=0x7fffffffdd80) at util/session.c:679
5 0x0000000000471c8d in __perf_session__process_events (session=0x915ca0, data_offset=400, data_size=150448, file_size=150848, tool=
0x7fffffffdd80) at util/session.c:1363
6 0x0000000000471d42 in perf_session__process_events (self=0x915ca0, tool=0x7fffffffdd80) at util/session.c:1379
7 0x000000000042484a in __cmd_report (rep=0x7fffffffdd80) at builtin-report.c:368
8 0x0000000000425bf1 in cmd_report (argc=0, argv=0x915b00, prefix=0x0) at builtin-report.c:756
9 0x0000000000438505 in __cmd_report (argc=4, argv=0x7fffffffe260) at builtin-kvm.c:8410 0x000000000043882a in cmd_kvm (argc=4, argv=0x7fffffffe260, prefix=0x0) at builtin-kvm.c:131 11 0x00000000004152cd in run_builtin (p=0x7a54e8, argc=9, argv=0x7fffffffe260) at perf.c:273 12 0x00000000004154c7 in handle_internal_command (argc=9, argv=0x7fffffffe260) at perf.c:345 13 0x0000000000415613 in run_argv (argcp=0x7fffffffe14c, argv=0x7fffffffe140) at perf.c:389 14 0x0000000000415899 in main (argc=9, argv=0x7fffffffe260) at perf.c:487
If no samples hit within a guest module then no build id events are written to the header during the record. On the report side this cases no 'machine' to be generated for the default guest (pid == 0) leading to a machine == NULL in perf_session_deliver_event.
perf's handling of default kallsyms and guest-modules needs to be reworked. Until then perf should at least not segfault.
--- tools/perf/util/session.c | 2 ++ 1 file changed, 2 insertions(+)
tools/perf/util/session.c +2 -0
@@ -976,6 +976,8 @@ static int perf_session_deliver_event(struct perf_session *session, }return tool->sample (tool, event, sample, evsel, machine);case PERF_RECORD_MMAP : +if (machine ==NULL ) +return 0 ;return tool->mmap (tool, event, sample, machine);case PERF_RECORD_COMM :return tool->comm (tool, event, sample, machine);
Signature · 1 line
1.7.10.1
Namhyung Kim
maintainerHi, David
Quoted 62 lines · David Ahern
(This time with actual backtrace; forgot to remove the leading '#' on the frames.)
File generated with:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules record -aThe report command:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules reportdies with a SEGFAULT:
Program received signal SIGSEGV, Segmentation fault.0x000000000046dd7binmachine__mmap_name(self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) atutil/map.c:715
715snprintf(bf, size, "[%s.%d]", "guest.kernel.kallsyms", self->pid);(gdb) bt
0 0x000000000046dd7b in machine__mmap_name (self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) at util/map.c:715 1 0x0000000000444161 in perf_event__process_kernel_mmap (tool=0x7fffffffdd80, event=0x7ffff7fb4120, machine=0x0) at util/event.c:562 2 0x0000000000444642 in perf_event__process_mmap (tool=0x7fffffffdd80, event=0x7ffff7fb4120, sample=0x7fffffffd210, machine=0x0) at util/event.c:668 3 0x0000000000470e0b in perf_session_deliver_event (session=0x915ca0, event=0x7ffff7fb4120, sample=0x7fffffffd210, tool=0x7fffffffdd80, file_offset=8480) at util/session.c:979 4 0x000000000047032e in flush_sample_queue (s=0x915ca0, tool=0x7fffffffdd80) at util/session.c:679 5 0x0000000000471c8d in __perf_session__process_events (session=0x915ca0, data_offset=400, data_size=150448, file_size=150848, tool= 0x7fffffffdd80) at util/session.c:1363 6 0x0000000000471d42 in perf_session__process_events (self=0x915ca0, tool=0x7fffffffdd80) at util/session.c:1379 7 0x000000000042484a in __cmd_report (rep=0x7fffffffdd80) at builtin-report.c:368 8 0x0000000000425bf1 in cmd_report (argc=0, argv=0x915b00, prefix=0x0) at builtin-report.c:756 9 0x0000000000438505 in __cmd_report (argc=4, argv=0x7fffffffe260) at builtin-kvm.c:8410
0x000000000043882aincmd_kvm(argc=4, argv=0x7fffffffe260, prefix=0x0) at builtin-kvm.c:131 110x00000000004152cdinrun_builtin(p=0x7a54e8, argc=9, argv=0x7fffffffe260) at perf.c:273 120x00000000004154c7inhandle_internal_command(argc=9, argv=0x7fffffffe260) at perf.c:345 130x0000000000415613inrun_argv(argcp=0x7fffffffe14c, argv=0x7fffffffe140) at perf.c:389 140x0000000000415899in main (argc=9, argv=0x7fffffffe260) at perf.c:487If no samples hit within a guest module then no build id events are written to the header during the record. On the report side this cases no 'machine' to be generated for the default guest (pid == 0) leading to a machine == NULL in
perf_session_deliver_event.perf's handling of default kallsyms and guest-modules needs to be reworked. Until then perf should at least not segfault.
Reported-by Pradeep Kumar Surisetty <[email protected]>CC Arnaldo Carvalho de Melo <[email protected]>Cc Ingo Molnar <[email protected]>Cc Jiri Olsa <[email protected]>Cc Namhyung Kim <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Peter Zijlstra <[email protected]>Signed-off-by David Ahern <[email protected]>Signed-off-by David Ahern <[email protected]>--- tools/perf/util/session.c | 2 ++ 1 file changed, 2 insertions(+)tools/perf/util/session.c +0
-0
tools/perf/util/session.c static int perf_session_deliver_event(struct perf_session *session, · line 983
Quoted 8 lines
tools/perf/util/session.c +2
-0@@ -976,6 +976,8 @@ static int perf_session_deliver_event(struct perf_session *session, }return tool->sample (tool, event, sample, evsel, machine);case PERF_RECORD_MMAP : +if (machine ==NULL ) +return 0 ;return tool->mmap (tool, event, sample, machine);case PERF_RECORD_COMM :return tool->comm (tool, event, sample, machine);
I don't know about the kvm, but isn't there a possibility that the same thing happends on other types of event too - i.e. comm, fork, exit..?
Thanks,
Namhyung
On 6/25/12 2:39 AM, Namhyung Kim wrote
Quoted 3 lines
I don't know about the kvm, but isn't there a possibility that the same thing happends on other types of event too - i.e. comm, fork, exit..?
COMM, FORK, EXIT events cannot be generated in the guest of a guest; host machine is always non-NULL.
David
Quoted 7 lines · David Ahern
On 6/25/12 2:39 AM, Namhyung Kim wrote
Quoted 3 lines
I don't know about the kvm, but isn't there a possibility that the same thing happends on other types of event too - i.e. comm, fork, exit..?
COMM, FORK, EXIT events cannot be generated in the guest of a guest; host machine is always non-NULL.
s/guest of a guest/context of a guest/
[That's I what get for rushing an email as I run out the door this morning.]
Namhyung Kim
maintainerQuoted 6 lines · David Ahern
Quoted 5 lines · David Ahern
On 6/25/12 2:39 AM, Namhyung Kim wrote
Quoted 3 lines
I don't know about the kvm, but isn't there a possibility that the same thing happends on other types of event too - i.e. comm, fork, exit..?
Quoted 6 lines
Quoted 2 lines
COMM, FORK, EXIT events cannot be generated in the guest of a guest; host machine is always non-NULL.
s/guest of a guest/context of a guest/
[That's I what get for rushing an email as I run out the door this morning.]
:) Thanks for the correction!
Thanks,
Namhyung
Quoted 35 lines · David Ahern
(This time with actual backtrace; forgot to remove the leading '#' on the frames.)
File generated with:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules record -aThe report command:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules reportdies with a SEGFAULT:
Program received signal SIGSEGV, Segmentation fault.0x000000000046dd7binmachine__mmap_name(self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) atutil/map.c:715
715snprintf(bf, size, "[%s.%d]", "guest.kernel.kallsyms", self->pid);(gdb) bt
0 0x000000000046dd7b in machine__mmap_name (self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) at util/map.c:715 1 0x0000000000444161 in perf_event__process_kernel_mmap (tool=0x7fffffffdd80, event=0x7ffff7fb4120, machine=0x0) at util/event.c:562 2 0x0000000000444642 in perf_event__process_mmap (tool=0x7fffffffdd80, event=0x7ffff7fb4120, sample=0x7fffffffd210, machine=0x0) at util/event.c:668 3 0x0000000000470e0b in perf_session_deliver_event (session=0x915ca0, event=0x7ffff7fb4120, sample=0x7fffffffd210, tool=0x7fffffffdd80, file_offset=8480) at util/session.c:979 4 0x000000000047032e in flush_sample_queue (s=0x915ca0, tool=0x7fffffffdd80) at util/session.c:679 5 0x0000000000471c8d in __perf_session__process_events (session=0x915ca0, data_offset=400, data_size=150448, file_size=150848, tool= 0x7fffffffdd80) at util/session.c:1363 6 0x0000000000471d42 in perf_session__process_events (self=0x915ca0, tool=0x7fffffffdd80) at util/session.c:1379 7 0x000000000042484a in __cmd_report (rep=0x7fffffffdd80) at builtin-report.c:368 8 0x0000000000425bf1 in cmd_report (argc=0, argv=0x915b00, prefix=0x0) at builtin-report.c:756 9 0x0000000000438505 in __cmd_report (argc=4, argv=0x7fffffffe260) at builtin-kvm.c:8410
0x000000000043882aincmd_kvm(argc=4, argv=0x7fffffffe260, prefix=0x0) at builtin-kvm.c:131 110x00000000004152cdinrun_builtin(p=0x7a54e8, argc=9, argv=0x7fffffffe260) at perf.c:273 120x00000000004154c7inhandle_internal_command(argc=9, argv=0x7fffffffe260) at perf.c:345 130x0000000000415613inrun_argv(argcp=0x7fffffffe14c, argv=0x7fffffffe140) at perf.c:389 140x0000000000415899in main (argc=9, argv=0x7fffffffe260) at perf.c:487
Quoted 6 lines
If no samples hit within a guest module then no build id events are written to the header during the record. On the report side this cases no 'machine' to be generated for the default guest (pid == 0) leading to a machine == NULL in
perf_session_deliver_event.perf's handling of default kallsyms and guest-modules needs to be reworked.
agreed ;) any plans on that?
jirka
Quoted 2 lines · Jiri Olsa
Quoted 1 line
perf's handling of default kallsyms and guest-modules needs to be reworked.
agreed ;) any plans on that?
I am working on it as time allows. After debugging this latest data point (running longer avoid the segfault) I certainly understand the problem better, but the problem keeps expanding too -- like wrong paths on guest side modules. It really suggests that this notion of a 'default' guest is just wrong - too little information. We'll see.
David
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 22 lines · Arnaldo Carvalho de Melo
Em Sun, Jun 24, 2012 at 10:41:26PM -0600, David Ahern escreveu:
Quoted 8 lines
File generated with:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules record -aThe report command:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules reportQuoted 8 lines
dies with a SEGFAULT:
Program received signal SIGSEGV, Segmentation fault.0x000000000046dd7binmachine__mmap_name(self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) atutil/map.c:715
715snprintf(bf, size, "[%s.%d]", "guest.kernel.kallsyms", self->pid);(gdb) bt
00x000000000046dd7binmachine__mmap_name(self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) atutil/map.c:71510x0000000000444161inperf_event__process_kernel_mmap(tool=0x7fffffffdd80, event=0x7ffff7fb4120, machine=0x0) atutil/event.c:562I think that this should be handled at
perf_event__process_kernel_mmap, i.e. tools can conceivably handle this situation in their specific handlers.
Quoted 10 lines
tools/perf/util/session.c static int perf_session_deliver_event(struct perf_session *session, · line 981
Quoted 8 lines
@@ -976,6 +976,8 @@ static int perf_session_deliver_event(struct
perf_session*session,
}
returntool->sample(tool, event, sample, evsel, machine);
casePERF_RECORD_MMAP:tools/perf/util/session.c +2
-0+if (machine ==NULL ) +return 0 ;return tool->mmap (tool, event, sample, machine);- Arnaldo
Ok. Doing it in perf_session_deliver_event makes it consistent with the SAMPLE case.
David
Quoted 22 lines · Arnaldo Carvalho de Melo
Em Sun, Jun 24, 2012 at 10:41:26PM -0600, David Ahern escreveu:
Quoted 8 lines
File generated with:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules record -aThe report command:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules reportQuoted 8 lines
dies with a SEGFAULT:
Program received signal SIGSEGV, Segmentation fault.0x000000000046dd7binmachine__mmap_name(self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) atutil/map.c:715
715snprintf(bf, size, "[%s.%d]", "guest.kernel.kallsyms", self->pid);(gdb) bt
00x000000000046dd7binmachine__mmap_name(self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) atutil/map.c:71510x0000000000444161inperf_event__process_kernel_mmap(tool=0x7fffffffdd80, event=0x7ffff7fb4120, machine=0x0) atutil/event.c:562I think that this should be handled at
perf_event__process_kernel_mmap, i.e. tools can conceivably handle this situation in their specific handlers.
Quoted 10 lines
tools/perf/util/session.c static int perf_session_deliver_event(struct perf_session *session, · line 981
Quoted 8 lines
@@ -976,6 +976,8 @@ static int perf_session_deliver_event(struct
perf_session*session,
}
returntool->sample(tool, event, sample, evsel, machine);
casePERF_RECORD_MMAP:tools/perf/util/session.c +2
-0+if (machine ==NULL ) +return 0 ;return tool->mmap (tool, event, sample, machine);- Arnaldo
Ok. Doing it in perf_session_deliver_event makes it consistent with the SAMPLE case.
David
Quoted 18 lines · Arnaldo Carvalho de Melo
Em Sun, Jun 24, 2012 at 10:41:26PM -0600, David Ahern escreveu:
Quoted 16 lines
File generated with:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules record -aThe report command:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules reportdies with a SEGFAULT:
Program received signal SIGSEGV, Segmentation fault.0x000000000046dd7binmachine__mmap_name(self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) atutil/map.c:715
715snprintf(bf, size, "[%s.%d]", "guest.kernel.kallsyms", self->pid);(gdb) bt
00x000000000046dd7binmachine__mmap_name(self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) atutil/map.c:71510x0000000000444161inperf_event__process_kernel_mmap(tool=0x7fffffffdd80, event=0x7ffff7fb4120, machine=0x0) atutil/event.c:562
So this is the root cause of the SEGFAULT:
743eb868657bdb1b26c7b24077ca21c67c82c777 is the first bad commit commit 743eb868657bdb1b26c7b24077ca21c67c82c777
Author: Arnaldo Carvalho de Melo <[email protected]> Date: Mon Nov 28 07:56:39 2011 -0200
perf tools: Resolve machine earlier and pass it to perf_event_ops
Reducing the exposure of perf_session further, so that we can use the classes in cases where no perf.data file is created.
Cc: David Ahern <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: Mike Galbraith <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephane Eranian <[email protected]>
Link:
http://lkml.kernel.org/n/[email protected]
:040000 040000 414f7e982a230f633440b3da2900da4d2d9c30ef 3231e148910bd596c61814e18bba77ecf7ad5dfb M tools
specifically, this part:
- machine = perf_session__findnew_machine(session, event->mmap.pid);
- if (!machine) {
- pr_err("Can't find id %d's machine\n", event->mmap.pid);
- goto out_problem;
- }
-
Suggestions on how to fix that part?
More to the perf-kvm story; details in future emails.
David
Arnaldo:
Please ignore this one in favor of the patch I just sent -- 'perf kvm: fix regression with guest machine creation'.
Quoted 71 lines · David Ahern
(This time with actual backtrace; forgot to remove the leading '#' on the frames.)
File generated with:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules record -aThe report command:
$ perf kvm --host --guest --guestkallsyms=/tmp/guest-kallsyms --guestmodules=/tmp/guest-modules reportdies with a SEGFAULT:
Program received signal SIGSEGV, Segmentation fault.0x000000000046dd7binmachine__mmap_name(self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) atutil/map.c:715
715snprintf(bf, size, "[%s.%d]", "guest.kernel.kallsyms", self->pid);(gdb) bt
0 0x000000000046dd7b in machine__mmap_name (self=0x0, bf=0x7fffffffbd20 "q\021", size=4096) at util/map.c:715 1 0x0000000000444161 in perf_event__process_kernel_mmap (tool=0x7fffffffdd80, event=0x7ffff7fb4120, machine=0x0) at util/event.c:562 2 0x0000000000444642 in perf_event__process_mmap (tool=0x7fffffffdd80, event=0x7ffff7fb4120, sample=0x7fffffffd210, machine=0x0) at util/event.c:668 3 0x0000000000470e0b in perf_session_deliver_event (session=0x915ca0, event=0x7ffff7fb4120, sample=0x7fffffffd210, tool=0x7fffffffdd80, file_offset=8480) at util/session.c:979 4 0x000000000047032e in flush_sample_queue (s=0x915ca0, tool=0x7fffffffdd80) at util/session.c:679 5 0x0000000000471c8d in __perf_session__process_events (session=0x915ca0, data_offset=400, data_size=150448, file_size=150848, tool= 0x7fffffffdd80) at util/session.c:1363 6 0x0000000000471d42 in perf_session__process_events (self=0x915ca0, tool=0x7fffffffdd80) at util/session.c:1379 7 0x000000000042484a in __cmd_report (rep=0x7fffffffdd80) at builtin-report.c:368 8 0x0000000000425bf1 in cmd_report (argc=0, argv=0x915b00, prefix=0x0) at builtin-report.c:756 9 0x0000000000438505 in __cmd_report (argc=4, argv=0x7fffffffe260) at builtin-kvm.c:8410
0x000000000043882aincmd_kvm(argc=4, argv=0x7fffffffe260, prefix=0x0) at builtin-kvm.c:131 110x00000000004152cdinrun_builtin(p=0x7a54e8, argc=9, argv=0x7fffffffe260) at perf.c:273 120x00000000004154c7inhandle_internal_command(argc=9, argv=0x7fffffffe260) at perf.c:345 130x0000000000415613inrun_argv(argcp=0x7fffffffe14c, argv=0x7fffffffe140) at perf.c:389 140x0000000000415899in main (argc=9, argv=0x7fffffffe260) at perf.c:487If no samples hit within a guest module then no build id events are written to the header during the record. On the report side this cases no 'machine' to be generated for the default guest (pid == 0) leading to a machine == NULL in
perf_session_deliver_event.perf's handling of default kallsyms and guest-modules needs to be reworked. Until then perf should at least not segfault.
Reported-by Pradeep Kumar Surisetty <[email protected]>CC Arnaldo Carvalho de Melo <[email protected]>Cc Ingo Molnar <[email protected]>Cc Jiri Olsa <[email protected]>Cc Namhyung Kim <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Peter Zijlstra <[email protected]>Signed-off-by David Ahern <[email protected]>Signed-off-by David Ahern <[email protected]>--- tools/perf/util/session.c | 2 ++ 1 file changed, 2 insertions(+)tools/perf/util/session.c +2
-0@@ -976,6 +976,8 @@ static int perf_session_deliver_event(struct perf_session *session, }return tool->sample (tool, event, sample, evsel, machine);case PERF_RECORD_MMAP : +if (machine ==NULL ) +return 0 ;return tool->mmap (tool, event, sample, machine);case PERF_RECORD_COMM :return tool->comm (tool, event, sample, machine);