perf tools: Fix buildid processing
Adrian Hunter
reviewercover perf tools: Fix buildid processing
Hi
Here are 3 patches related to the "getting stuck at end" problem.
The first patch dodges the deadlock.
The second patch stops fork from messing up i.e.
prevents the deadlock condition from arising in
the first place.
The third patch processes the data in order i.e.
prevents fork from having to deal with weird data
anyway.
Adrian Hunter (3):
perf tools: Avoid deadlock when map_groups are broken perf tools: Make fork event processing more resilient perf tools: Fix buildid processing
tools/perf/builtin-record.c | 11 +++++++++++ tools/perf/util/machine.c | 20 ++++++++++++++++++-- tools/perf/util/thread.c | 6 ++++++ 3 files changed, 35 insertions(+), 2 deletions(-)
Regards
Adrian
Adrian Hunter
reviewerPATCH 1/3 perf tools: Avoid deadlock when map_groups are broken
Attempting to clone map groups onto themselves will deadlock. It only happens because of other bugs, but the code should protect itself anyway.
--- tools/perf/util/thread.c | 6 ++++++ 1 file changed, 6 insertions(+)
tools/perf/util/thread.c +6 -0
@@ -191,6 +191,12 @@ static int thread__clone_map_groups(struct thread *thread,if (thread->pid_ == parent->pid_)return 0 ; +if (thread->mg == parent->mg) { +dump_printf ("broken map groups on thread %d/%d parent %d/%d\n" , + thread->pid_, thread->tid, parent->pid_, parent->tid); +return 0 ; + } +/* But this one is new process, copy maps. */ for (i =0 ; i <MAP__NR_TYPES ; ++i)if (map_groups__clone (thread->mg, parent->mg, i) <0 )
Signature · 1 line
1.9.1
tip-bot for Adrian Hunter
merge bot[tip:perf/core] perf tools: Avoid deadlock when map_groups are broken
perf tools: Avoid deadlock when map_groups are broken
Commit-ID: 0d7e7acc47133e82911e24758178815cfee309e9 Gitweb: http://git.kernel.org/tip/0d7e7acc47133e82911e24758178815cfee309e9 Author: Adrian Hunter <[email protected]> AuthorDate: Wed, 19 Aug 2015 17:29:19 +0300 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Wed, 19 Aug 2015 14:15:09 -0300
perf tools: Avoid deadlock when map_groups are broken
Attempting to clone map groups onto themselves will deadlock.
It only happens because of other bugs, but the code should protect itself anyway.
--- tools/perf/util/thread.c | 6 ++++++ 1 file changed, 6 insertions(+)
tools/perf/util/thread.c +6 -0
@@ -191,6 +191,12 @@ static int thread__clone_map_groups(struct thread *thread,if (thread->pid_ == parent->pid_)return 0 ; +if (thread->mg == parent->mg) { +pr_debug ("broken map groups on thread %d/%d parent %d/%d\n" , + thread->pid_, thread->tid, parent->pid_, parent->tid); +return 0 ; + } +/* But this one is new process, copy maps. */ for (i =0 ; i <MAP__NR_TYPES ; ++i)if (map_groups__clone (thread->mg, parent->mg, i) <0 )
Adrian Hunter
reviewerPATCH 2/3 perf tools: Make fork event processing more resilient
In reply to Adrian HunterWhen processing a fork event, the tools lookup the parent thread by its tid. In a couple of cases, it is possible for that thread to have the wrong pid. That can happen if the data is being processed out of order, or if the (fork) event that would have removed the erroneous thread was lost. Assume the latter case, print a dump message, remove the erroneous thread, create a new one with the correct pid, and keep going.
--- tools/perf/util/machine.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)
tools/perf/util/machine.c +18 -2
@@ -1387,6 +1387,24 @@ int machine__process_fork_event(struct machine *machine, union perf_event *event event->fork.ptid);int err =0 ; +if (dump_trace) +perf_event__fprintf_task (event, stdout); + +/* +* There may be an existing thread that is not actually the parent, +* either because we are processing events out of order, or because the +* (fork) event that would have removed the thread was lost. Assume the +* latter case and continue on as best we can. +*/ +if (parent->pid_ != (pid_t )event->fork.ppid) { +dump_printf ("removing erroneous parent thread %d/%d\n" , + parent->pid_, parent->tid); +machine__remove_thread (machine, parent); +thread__put (parent); + parent =machine__findnew_thread (machine, event->fork.ppid, + event->fork.ptid); + } +/* if a thread currently exists for the thread id remove it */ if (thread !=NULL ) {machine__remove_thread (machine, thread);
@@ -1395,8 +1413,6 @@ int machine__process_fork_event(struct machine *machine, union perf_event *event thread =machine__findnew_thread (machine, event->fork.pid, event->fork.tid); -if (dump_trace) -perf_event__fprintf_task (event, stdout);if (thread ==NULL || parent ==NULL ||thread__fork (thread, parent, sample->time) <0 ) {
Signature · 1 line
1.9.1
tip-bot for Adrian Hunter
merge bot[tip:perf/core] perf tools: Make fork event processing more resilient
perf tools: Make fork event processing more resilient
Commit-ID: 5cb73340d92a716fd2776700742c3558206ae298 Gitweb: http://git.kernel.org/tip/5cb73340d92a716fd2776700742c3558206ae298 Author: Adrian Hunter <[email protected]> AuthorDate: Wed, 19 Aug 2015 17:29:20 +0300 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Wed, 19 Aug 2015 14:15:25 -0300
perf tools: Make fork event processing more resilient
When processing a fork event, the tools lookup the parent thread by its tid. In a couple of cases, it is possible for that thread to have the wrong pid.
That can happen if the data is being processed out of order, or if the (fork) event that would have removed the erroneous thread was lost.
Assume the latter case, print a dump message, remove the erroneous thread, create a new one with the correct pid, and keep going.
--- tools/perf/util/machine.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)
tools/perf/util/machine.c +18 -2
@@ -1387,6 +1387,24 @@ int machine__process_fork_event(struct machine *machine, union perf_event *event event->fork.ptid);int err =0 ; +if (dump_trace) +perf_event__fprintf_task (event, stdout); + +/* +* There may be an existing thread that is not actually the parent, +* either because we are processing events out of order, or because the +* (fork) event that would have removed the thread was lost. Assume the +* latter case and continue on as best we can. +*/ +if (parent->pid_ != (pid_t )event->fork.ppid) { +dump_printf ("removing erroneous parent thread %d/%d\n" , + parent->pid_, parent->tid); +machine__remove_thread (machine, parent); +thread__put (parent); + parent =machine__findnew_thread (machine, event->fork.ppid, + event->fork.ptid); + } +/* if a thread currently exists for the thread id remove it */ if (thread !=NULL ) {machine__remove_thread (machine, thread);
@@ -1395,8 +1413,6 @@ int machine__process_fork_event(struct machine *machine, union perf_event *event thread =machine__findnew_thread (machine, event->fork.pid, event->fork.tid); -if (dump_trace) -perf_event__fprintf_task (event, stdout);if (thread ==NULL || parent ==NULL ||thread__fork (thread, parent, sample->time) <0 ) {
Adrian Hunter
reviewerPATCH 3/3 perf tools: Fix buildid processing
In reply to Adrian HunterAfter recording, 'perf record' post-processes the data to determine which buildids are needed. That processing must process the data in time order, if possible, because otherwise dependent events, like forks and mmaps, will not make sense.
--- tools/perf/builtin-record.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
tools/perf/builtin-record.c +11 -0
@@ -500,6 +500,15 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)return -1 ; } +/* +* Normally perf_session__new would do this, but it doesn't have the +* evlist. +*/ +if (rec->tool.ordered_events && !perf_evlist__sample_id_all (rec->evlist)) { +dump_printf ("WARNING: No sample_id_all support, falling back to unordered processing\n" ); + rec->tool.ordered_events = false; + } + fd =perf_data_file__fd (file); rec->session = session;
@@ -965,9 +974,11 @@ static struct record record = { .tool = { .sample = process_sample_event, .fork = perf_event__process_fork, + .exit = perf_event__process_exit, .comm = perf_event__process_comm, .mmap = perf_event__process_mmap, .mmap2 = perf_event__process_mmap2, + .ordered_events = true, }, };
Signature · 1 line
1.9.1
Adrian Hunter
reviewerQuoted 18 lines · Adrian Hunter
After recording, 'perf record' post-processes the data to determine which buildids are needed. That processing must process the data in time order, if possible, because otherwise dependent events, like forks and mmaps, will not make sense.
Signed-off-by Adrian Hunter <[email protected]>--- tools/perf/builtin-record.c | 11 +++++++++++ 1 file changed, 11 insertions(+)tools/perf/builtin-record.c +0
-0@@ -500,6 +500,15 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)return -1 ; }
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 511
Quoted 10 lines
tools/perf/builtin-record.c +9
-0+/* +* Normally perf_session__new would do this, but it doesn't have the +* evlist. +*/ +if (rec->tool.ordered_events && !perf_evlist__sample_id_all (rec->evlist)) { +dump_printf ("WARNING: No sample_id_all support, falling back to unordered processing\n" ); + rec->tool.ordered_events = false; + } +
In fact this chunk is misplaced, it needs to be below the call to record__open() after which the attributes are correctly configured. And dump_printf() is no good for 'perf record', pr_debug would be better. Don't have time to do that right now, I'll do it tommorow unless Arnaldo fixes it up.
Quoted 16 lines
tools/perf/builtin-record.c +2
-0fd =perf_data_file__fd (file); rec->session = session;@@ -965,9 +974,11 @@ static struct record record = { .tool = { .sample = process_sample_event, .fork = perf_event__process_fork, + .exit = perf_event__process_exit, .comm = perf_event__process_comm, .mmap = perf_event__process_mmap, .mmap2 = perf_event__process_mmap2, + .ordered_events = true, }, };
Arnaldo Carvalho de Melo
maintainerEm Wed, Aug 19, 2015 at 06:23:51PM +0300, Adrian Hunter escreveu:
Quoted 20 lines · Adrian Hunter
Quoted 19 lines
After recording, 'perf record' post-processes the data to determine which buildids are needed. That processing must process the data in time order, if possible, because otherwise dependent events, like forks and mmaps, will not make sense.
Signed-off-by Adrian Hunter <[email protected]>--- tools/perf/builtin-record.c | 11 +++++++++++ 1 file changed, 11 insertions(+)tools/perf/builtin-record.c +0
-0@@ -500,6 +500,15 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)return -1 ; }
Quoted 14 lines
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 511
Quoted 9 lines
tools/perf/builtin-record.c +9
-0+/* +* Normally perf_session__new would do this, but it doesn't have the +* evlist. +*/ +if (rec->tool.ordered_events && !perf_evlist__sample_id_all (rec->evlist)) { +dump_printf ("WARNING: No sample_id_all support, falling back to unordered processing\n" ); + rec->tool.ordered_events = false; + } +In fact this chunk is misplaced, it needs to be below the call to
record__open()after which the attributes are correctly configured. Anddump_printf()is no good for 'perf record',pr_debugwould be better. Don't have time to do that right now, I'll do it tommorow unless Arnaldo fixes it up.
I'll fix it up, thanks!
- Arnaldo
Quoted 16 lines
Quoted 16 lines
tools/perf/builtin-record.c +2
-0fd =perf_data_file__fd (file); rec->session = session;@@ -965,9 +974,11 @@ static struct record record = { .tool = { .sample = process_sample_event, .fork = perf_event__process_fork, + .exit = perf_event__process_exit, .comm = perf_event__process_comm, .mmap = perf_event__process_mmap, .mmap2 = perf_event__process_mmap2, + .ordered_events = true, }, };
tip-bot for Adrian Hunter
merge botperf tools: Fix buildid processing
Commit-ID: cca8482c0651cea97aade58cc22109ce9fffbfa2 Gitweb: http://git.kernel.org/tip/cca8482c0651cea97aade58cc22109ce9fffbfa2 Author: Adrian Hunter <[email protected]> AuthorDate: Wed, 19 Aug 2015 17:29:21 +0300 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Wed, 19 Aug 2015 14:15:26 -0300
perf tools: Fix buildid processing
After recording, 'perf record' post-processes the data to determine which buildids are needed.
That processing must process the data in time order, if possible, because otherwise dependent events, like forks and mmaps, will not make sense.
--- tools/perf/builtin-record.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
tools/perf/builtin-record.c +11 -0
@@ -521,6 +521,15 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) goto out_child; } +/* +* Normally perf_session__new would do this, but it doesn't have the +* evlist. +*/ +if (rec->tool.ordered_events && !perf_evlist__sample_id_all (rec->evlist)) { +pr_warning ("WARNING: No sample_id_all support, falling back to unordered processing\n" ); + rec->tool.ordered_events = false; + } +if (!rec->evlist->nr_groups)perf_header__clear_feat (&session->header,HEADER_GROUP_DESC );
@@ -965,9 +974,11 @@ static struct record record = { .tool = { .sample = process_sample_event, .fork = perf_event__process_fork, + .exit = perf_event__process_exit, .comm = perf_event__process_comm, .mmap = perf_event__process_mmap, .mmap2 = perf_event__process_mmap2, + .ordered_events = true, }, };
Quoted 6 lines · Adrian Hunter
Hi
Here are 3 patches related to the "getting stuck at end" problem.
The first patch dodges the deadlock.
Quoted 7 lines
The second patch stops fork from messing up i.e.
prevents the deadlock condition from arising in
the first place.The third patch processes the data in order i.e.
prevents fork from having to deal with weird data
anyway.
Tested/Acked-by: Jiri Olsa <[email protected]>
thanks,
jirka