Skip to contentSkip to conversation

perf: Speed up thread map generation

PATCH Superseded Don Zickus 2 participants 3 messages Mar 13, 2014 – Mar 14, 2014 perf events

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Don Zickus

author
#0 lore raw

PATCH 1/1 perf: Speed up thread map generation

When trying to capture perf data on a system running spejbb2013, perf hung for about 15 minutes. This is because it took that long to gather about 10,000 thread maps and process them.

I don't think a user wants to wait that long.

Instead, recognize that thread maps are roughly equivalent to pid maps and just quickly copy those instead.

To do this, I synthesize 'fork' events, this eventually calls thread__fork() and copies the maps over.

The overhead goes from 15 minutes down to about a few seconds.

---
 tools/perf/util/event.c |   39 ++++++++++++++++++++++++++++++++++++---
 1 files changed, 36 insertions(+), 3 deletions(-)

---
Based on perf/core: 1c075d114d0f3be

tools/perf/util/event.c +36 -3
index 55eebe9..95e5649 100644
@@ -129,6 +129,28 @@ out:
 	return tgid;
 }
 
+static int perf_event__synthesize_fork(struct perf_tool *tool,
+				       union perf_event *event, pid_t pid,
+				       pid_t tgid, perf_event__handler_t process,
+				       struct machine *machine)
+{
+	memset(&event->fork, 0, sizeof(event->fork) + machine->id_hdr_size);
+
+	/* this is really a clone event but we use fork to synthesize it */
+	event->fork.ppid = tgid;
+	event->fork.ptid = tgid;
+	event->fork.pid  = tgid;
+	event->fork.tid  = pid;
+	event->fork.header.type = PERF_RECORD_FORK;
+
+	event->fork.header.size = (sizeof(event->fork) + machine->id_hdr_size);
+
+	if (process(tool, event, &synth_sample, machine) != 0)
+		return -1;
+
+	return 0;
+}
+
 int perf_event__synthesize_mmap_events(struct perf_tool *tool,
 				       union perf_event *event,
 				       pid_t pid, pid_t tgid,
@@ -287,6 +309,11 @@ static int __event__synthesize_thread(union perf_event *comm_event,
 	DIR *tasks;
 	struct dirent dirent, *next;
 	pid_t tgid;
+	union perf_event *fork_event;
+
+	fork_event = malloc(sizeof(fork_event->fork) + machine->id_hdr_size);
+	if (fork_event == NULL)
+		return -1;
 
 	/* special case: only send one comm event using passed in pid */
 	if (!full) {
@@ -326,9 +353,15 @@ static int __event__synthesize_thread(union perf_event *comm_event,
 		if (tgid == -1)
 			return -1;
 
-		/* process the thread's maps too */
-		rc = perf_event__synthesize_mmap_events(tool, mmap_event, _pid, tgid,
-							process, machine, mmap_data);
+		if (_pid == pid) {
+			/* process the parent's maps too */
+			rc = perf_event__synthesize_mmap_events(tool, mmap_event, pid, tgid,
+						process, machine, mmap_data);
+		} else {
+			/* only fork the tid's map, to save time */
+			rc = perf_event__synthesize_fork(tool, fork_event, _pid, tgid,
+						 process, machine);
+		}
 
 		if (rc)
 			return rc;
Signature · 1 line
1.7.1
#1 lore raw

On Thu, Mar 13, 2014 at 05:16:37PM -0400, Don Zickus wrote:

SNIP

Quoted 17 lines
tools/perf/util/event.c +13 -0
+	event->fork.ptid = tgid;
+	event->fork.pid  = tgid;
+	event->fork.tid  = pid;
+	event->fork.header.type = PERF_RECORD_FORK;
+
+	event->fork.header.size = (sizeof(event->fork) + machine->id_hdr_size);
+
+	if (process(tool, event, &synth_sample, machine) != 0)
+		return -1;
+
+	return 0;
+}
+
 int perf_event__synthesize_mmap_events(struct perf_tool *tool,
 				       union perf_event *event,
 				       pid_t pid, pid_t tgid,

tools/perf/util/event.c static int __event__synthesize_thread(union perf_event *comm_event, · line 316

Quoted 8 lines
tools/perf/util/event.c +5 -0
@@ -287,6 +309,11 @@ static int __event__synthesize_thread(union perf_event *comm_event,
 	DIR *tasks;
 	struct dirent dirent, *next;
 	pid_t tgid;
+	union perf_event *fork_event;
+
+	fork_event = malloc(sizeof(fork_event->fork) + machine->id_hdr_size);
+	if (fork_event == NULL)
+		return -1;

need to be freed

maybe we could move the allocation one level up,
same as for mmap_event and comm_event

otherwise I think the logic with FORK event is correct

jirka

author
#2 lore raw
Quoted 23 lines · Jiri Olsa

On Thu, Mar 13, 2014 at 05:16:37PM -0400, Don Zickus wrote:

SNIP

Quoted 19 lines
tools/perf/util/event.c +13 -0
+	event->fork.ptid = tgid;
+	event->fork.pid  = tgid;
+	event->fork.tid  = pid;
+	event->fork.header.type = PERF_RECORD_FORK;
+
+	event->fork.header.size = (sizeof(event->fork) + machine->id_hdr_size);
+
+	if (process(tool, event, &synth_sample, machine) != 0)
+		return -1;
+
+	return 0;
+}
+
 int perf_event__synthesize_mmap_events(struct perf_tool *tool,
 				       union perf_event *event,
 				       pid_t pid, pid_t tgid,
@@ -287,6 +309,11 @@ static int __event__synthesize_thread(union perf_event *comm_event,
 	DIR *tasks;
 	struct dirent dirent, *next;
Quoted 8 lines

tools/perf/util/event.c static int __event__synthesize_thread(union perf_event *comm_event, · line 316

Quoted 6 lines
tools/perf/util/event.c +5 -0
 	pid_t tgid;
+	union perf_event *fork_event;
+
+	fork_event = malloc(sizeof(fork_event->fork) + machine->id_hdr_size);
+	if (fork_event == NULL)
+		return -1;

need to be freed

Doh.

Quoted 3 lines

maybe we could move the allocation one level up,
same as for mmap_event and comm_event

I thought about that too, just seemed silly to pass that through the function params. Not sure if there are different mallocs from the callers.

I'll respin the patch with this though.

Quoted 2 lines

otherwise I think the logic with FORK event is correct

Great!

Cheers,
Don