Skip to contentSkip to conversation

perf/live buildid

RFC PATCH In review Tom Zanussi 2 participants 4 messages May 1, 2010 perf events

lore
Share & tools
lore
#0 lore raw

cover perf/live buildid

The first patch in this patchset removes the current live-mode build id support, which probably makes sense to apply independent of the second.

The second is an RFC patch whose immediate purpose is to replace the current build-id support in 'live mode' with something that makes more sense for live mode - injecting the build ids before the samples that use them.

To do that, a new perf command, perf-inject, is introduced.

perf-inject basically pipes the stdout received from live-mode perf-record to the stdin of live-mode perf-report, but in-between provides a means for the event stream to be interpreted and augmented with 'injected events'.

In this case, the injected events are the same build-ids recorded in the current version, but provided in a more timely fashion.

perf-inject is meant to be a live-mode-only utility and normally provides no output other than the stdout piping that is its purpose; the output below uses the -v options on both perf-inject and perf-report to show what it's basically doing.

I haven't had time to try it out yet, but the idea would be that if you had the relevant build-ids on a remote system, say via perf-archive, you'd be able to run a live-mode perf session on one system and have the output displayed live on the remote system.

Currently perf-inject is used only to provide build-ids, but it should be easy to add support for other injected events e.g. it could be used to look up and create uid->user name events, or syscall id->syscall name events.

root@tropicana:~/perf-scripts# perf record -o - ./hackbench 10 | perf inject -v -b | perf report -v -i -

Running with 10*40 (== 400) tasks.
Time: 0.696
Looking at the vmlinux_path (5 entries long)
Using /lib/modules/2.6.34-rc5/build/vmlinux for symbols
build id found for /lib/modules/2.6.34-rc5/build/vmlinux: a925642e28500d4aeaffc62b2fc9071641ba303c
build id event received for /lib/modules/2.6.34-rc5/build/vmlinux: a925642e28500d4aeaffc62b2fc9071641ba303c Looking at the vmlinux_path (5 entries long)
no build_id found for /lib/libc-2.8.90.so
build id found for ./hackbench: 1f1a90c20ffd00527b6002483df73a96540f041e
no build_id found for /lib/libpthread-2.8.90.so
Using /lib/modules/2.6.34-rc5/build/vmlinux for symbols
build id event received for ./hackbench: 1f1a90c20ffd00527b6002483df73a96540f041e
build id found for /lib/modules/2.6.34-rc5/kernel/drivers/net/forcedeth.ko: fdeed20f17ce5ba3c12d9a0f5d9add57db7e8d3f build id event received for /lib/modules/2.6.34-rc5/kernel/drivers/net/forcedeth.ko: fdeed20f17ce5ba3c12d9a0f5d9add57db7e8d3f no build_id found for /lib/ld-2.8.90.so
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.121 MB - (~5299 samples) ]
# Samples: 2019104685
#
# Overhead Command Shared Object Symbol

# ........  .........  .......................................................  ......
#
     8.22%  hackbench  /lib/modules/2.6.34-rc5/build/vmlinux                    0xffffffff811fbb5d ! [k] copy_user_generic_string
     7.12%  hackbench  /lib/modules/2.6.34-rc5/build/vmlinux                    0xffffffff8111ef55 ! [k] kfree

Tom Zanussi (2):
perf/live: don't synthesize build ids at the end of a live mode trace
perf: add perf-inject builtin

 tools/perf/Makefile                |    1 +
 tools/perf/builtin-annotate.c      |    2 +-
 tools/perf/builtin-buildid-list.c  |    2 +-
 tools/perf/builtin-diff.c          |    4 +-
 tools/perf/builtin-inject.c        |  228 ++++++++++++++++++++++++++++++++++++
 tools/perf/builtin-kmem.c          |    2 +-
 tools/perf/builtin-lock.c          |    2 +-
 tools/perf/builtin-record.c        |    9 +--
 tools/perf/builtin-report.c        |    2 +-
 tools/perf/builtin-sched.c         |    2 +-
 tools/perf/builtin-timechart.c     |    2 +-
 tools/perf/builtin-top.c           |    2 +-
 tools/perf/builtin-trace.c         |    2 +-
 tools/perf/builtin.h               |    1 +
 tools/perf/perf.c                  |    1 +
 tools/perf/util/header.c           |   96 +++++-----------
 tools/perf/util/header.h           |    2 -
 tools/perf/util/session.c          |    3 +-
 tools/perf/util/session.h          |    3 +-
 tools/perf/util/trace-event-read.c |   19 +++-
 tools/perf/util/trace-event.h      |    2 +-
 21 files changed, 293 insertions(+), 94 deletions(-)
 create mode 100644 tools/perf/builtin-inject.c
#1 lore raw

RFC PATCH 1/2 perf/live: don't synthesize build ids at the end of a live mode trace

It doesn't really make sense to record the build ids at the end of a live mode session - live mode samples need that information during the trace rather than at the end.

Leave event__synthesize_build_id() in place, however; we'll still be using that to synthesize build ids in a more timely fashion in a future patch.

---
 tools/perf/builtin-record.c |    7 -----
 tools/perf/util/header.c    |   61 -------------------------------------------
 tools/perf/util/header.h    |    2 -
 3 files changed, 0 insertions(+), 70 deletions(-)
tools/perf/builtin-record.c +0 -7
index 722b6f1..5d21a3a 100644
@@ -446,13 +446,6 @@ static void atexit_header(void)
 
 		process_buildids();
 		perf_header__write(&session->header, output, true);
-	} else {
-		int err;
-
-		err = event__synthesize_build_ids(process_synthesized_event,
-						  session);
-		if (err < 0)
-			pr_err("Couldn't synthesize build ids.\n");
 	}
 }
 
tools/perf/util/header.c +0 -61

67 lines not on this page · raw message

tools/perf/util/header.h +0 -2
index f39443d..402ac24 100644
@@ -122,8 +122,6 @@ int event__synthesize_build_id(struct dso *pos, u16 misc,
 			       event__handler_t process,
 			       struct machine *machine,
 			       struct perf_session *session);
-int event__synthesize_build_ids(event__handler_t process,
-				struct perf_session *session);
 int event__process_build_id(event_t *self, struct perf_session *session);
 
 #endif /* __PERF_HEADER_H */
Signature · 1 line
1.6.4.GIT
#2 lore raw

RFC PATCH 2/2 perf: add perf-inject builtin

In reply to Tom Zanussi

Currently, perf 'live mode' writes build-ids at the end of the session, which isn't actually useful for processing live mode events.

What would be better would be to have the build-ids sent before any of the samples that reference them, which can be done by processing the event stream and retrieving the build-ids on the first hit. Doing that in perf-record itself, however, is off-limits.

This patch introduces perf-inject, which does the same job while leaving perf-record untouched. Normal mode perf still records the build-ids at the end of the session as it should, but for live mode, perf-inject can be injected in between the record and report steps e.g.:

perf record -o - ./hackbench 10 | perf inject -v -b | perf report -v -i -

perf-inject reads a perf-record event stream and repipes it to stdout. At any point the processing code can inject other events into the event stream - in this case build-ids (-b option) are read and injected as needed into the event stream.

Build-ids are just the first user of perf-inject - potentially anything that needs userspace processing to augment the trace stream with additional information could make use of this facility.

---
 tools/perf/Makefile                |    1 +
 tools/perf/builtin-annotate.c      |    2 +-
 tools/perf/builtin-buildid-list.c  |    2 +-
 tools/perf/builtin-diff.c          |    4 +-
 tools/perf/builtin-inject.c        |  228 ++++++++++++++++++++++++++++++++++++
 tools/perf/builtin-kmem.c          |    2 +-
 tools/perf/builtin-lock.c          |    2 +-
 tools/perf/builtin-record.c        |    2 +-
 tools/perf/builtin-report.c        |    2 +-
 tools/perf/builtin-sched.c         |    2 +-
 tools/perf/builtin-timechart.c     |    2 +-
 tools/perf/builtin-top.c           |    2 +-
 tools/perf/builtin-trace.c         |    2 +-
 tools/perf/builtin.h               |    1 +
 tools/perf/perf.c                  |    1 +
 tools/perf/util/header.c           |   35 +++++--
 tools/perf/util/session.c          |    3 +-
 tools/perf/util/session.h          |    3 +-
 tools/perf/util/trace-event-read.c |   19 +++-
 tools/perf/util/trace-event.h      |    2 +-
 20 files changed, 293 insertions(+), 24 deletions(-)
 create mode 100644 tools/perf/builtin-inject.c
tools/perf/Makefile +1 -0
index 739c441..73a9666 100644
@@ -491,6 +491,7 @@ BUILTIN_OBJS += $(OUTPUT)builtin-kmem.o
 BUILTIN_OBJS += $(OUTPUT)builtin-lock.o
 BUILTIN_OBJS += $(OUTPUT)builtin-kvm.o
 BUILTIN_OBJS += $(OUTPUT)builtin-test.o
+BUILTIN_OBJS += $(OUTPUT)builtin-inject.o
 
 PERFLIBS = $(LIB_FILE)
 
tools/perf/builtin-annotate.c +1 -1
index b57dbcf..ee154b5 100644
@@ -554,7 +554,7 @@ static int __cmd_annotate(void)
 	int ret;
 	struct perf_session *session;
 
-	session = perf_session__new(input_name, O_RDONLY, force);
+	session = perf_session__new(input_name, O_RDONLY, force, false);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-buildid-list.c +1 -1
index 7dc3b2e..44a47e1 100644
@@ -39,7 +39,7 @@ static int __cmd_buildid_list(void)
 	int err = -1;
 	struct perf_session *session;
 
-	session = perf_session__new(input_name, O_RDONLY, force);
+	session = perf_session__new(input_name, O_RDONLY, force, false);
 	if (session == NULL)
 		return -1;
 
tools/perf/builtin-diff.c +2 -2
index 207e860..4cce68f 100644
@@ -156,8 +156,8 @@ static int __cmd_diff(void)
 	int ret, i;
 	struct perf_session *session[2];
 
-	session[0] = perf_session__new(input_old, O_RDONLY, force);
-	session[1] = perf_session__new(input_new, O_RDONLY, force);
+	session[0] = perf_session__new(input_old, O_RDONLY, force, false);
+	session[1] = perf_session__new(input_new, O_RDONLY, force, false);
 	if (session[0] == NULL || session[1] == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-inject.c new file +228 -0

228 lines not on this page · raw message

tools/perf/builtin-kmem.c +1 -1
index ee05dba..31f60a2 100644
@@ -492,7 +492,7 @@ static void sort_result(void)
 static int __cmd_kmem(void)
 {
 	int err = -EINVAL;
-	struct perf_session *session = perf_session__new(input_name, O_RDONLY, 0);
+	struct perf_session *session = perf_session__new(input_name, O_RDONLY, 0, false);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-lock.c +1 -1
index ce27675..6605000 100644
@@ -818,7 +818,7 @@ static struct perf_event_ops eops = {
 
 static int read_events(void)
 {
-	session = perf_session__new(input_name, O_RDONLY, 0);
+	session = perf_session__new(input_name, O_RDONLY, 0, false);
 	if (!session)
 		die("Initializing perf session failed\n");
 
tools/perf/builtin-record.c +1 -1
index 5d21a3a..8364e00 100644
@@ -548,7 +548,7 @@ static int __cmd_record(int argc, const char **argv)
 	}
 
 	session = perf_session__new(output_name, O_WRONLY,
-				    write_mode == WRITE_FORCE);
+				    write_mode == WRITE_FORCE, false);
 	if (session == NULL) {
 		pr_err("Not enough memory for reading perf file header\n");
 		return -1;
tools/perf/builtin-report.c +1 -1
index f1b46eb..0152b54 100644
@@ -289,7 +289,7 @@ static int __cmd_report(void)
 
 	signal(SIGINT, sig_handler);
 
-	session = perf_session__new(input_name, O_RDONLY, force);
+	session = perf_session__new(input_name, O_RDONLY, force, false);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-sched.c +1 -1
index 94453f1..aef6ed0 100644
@@ -1660,7 +1660,7 @@ static struct perf_event_ops event_ops = {
 static int read_events(void)
 {
 	int err = -EINVAL;
-	struct perf_session *session = perf_session__new(input_name, O_RDONLY, 0);
+	struct perf_session *session = perf_session__new(input_name, O_RDONLY, 0, false);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-timechart.c +1 -1
index c35aa44..5a52ed9 100644
@@ -936,7 +936,7 @@ static struct perf_event_ops event_ops = {
 
 static int __cmd_timechart(void)
 {
-	struct perf_session *session = perf_session__new(input_name, O_RDONLY, 0);
+	struct perf_session *session = perf_session__new(input_name, O_RDONLY, 0, false);
 	int ret = -EINVAL;
 
 	if (session == NULL)
tools/perf/builtin-top.c +1 -1
index d95281f..3de3977 100644
@@ -1287,7 +1287,7 @@ static int __cmd_top(void)
 	 * FIXME: perf_session__new should allow passing a O_MMAP, so that all this
 	 * mmap reading, etc is encapsulated in it. Use O_WRONLY for now.
 	 */
-	struct perf_session *session = perf_session__new(NULL, O_WRONLY, false);
+	struct perf_session *session = perf_session__new(NULL, O_WRONLY, false, false);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-trace.c +1 -1
index 77f556f..9c483e9 100644
@@ -661,7 +661,7 @@ int cmd_trace(int argc, const char **argv, const char *prefix __used)
 	if (!script_name)
 		setup_pager();
 
-	session = perf_session__new(input_name, O_RDONLY, 0);
+	session = perf_session__new(input_name, O_RDONLY, 0, false);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin.h +1 -0
index 34a8a9a..921245b 100644
@@ -34,5 +34,6 @@ extern int cmd_kmem(int argc, const char **argv, const char *prefix);
 extern int cmd_lock(int argc, const char **argv, const char *prefix);
 extern int cmd_kvm(int argc, const char **argv, const char *prefix);
 extern int cmd_test(int argc, const char **argv, const char *prefix);
+extern int cmd_inject(int argc, const char **argv, const char *prefix);
 
 #endif
tools/perf/perf.c +1 -0
index 5ff9b5b..08e0e5d 100644
@@ -309,6 +309,7 @@ static void handle_internal_command(int argc, const char **argv)
 		{ "lock",	cmd_lock,	0 },
 		{ "kvm",	cmd_kvm,	0 },
 		{ "test",	cmd_test,	0 },
+		{ "inject",	cmd_inject,	0 },
 	};
 	unsigned int i;
 	static const char ext[] = STRIP_EXTENSION;
tools/perf/util/header.c +27 -8
index 83b79e2..2b9f898 100644
@@ -712,10 +712,18 @@ static int __event_process_build_id(struct build_id_event *bev,
 
 	dso = __dsos__findnew(head, filename);
 	if (dso != NULL) {
+		char sbuild_id[BUILD_ID_SIZE * 2 + 1];
+
 		dso__set_build_id(dso, &bev->build_id);
-			if (filename[0] == '[')
-				dso->kernel = dso_type;
-		}
+
+		if (filename[0] == '[')
+			dso->kernel = dso_type;
+
+		build_id__sprintf(dso->build_id, sizeof(dso->build_id),
+				  sbuild_id);
+		pr_debug("build id event received for %s: %s\n",
+			 dso->long_name, sbuild_id);
+	}
 
 	err = 0;
 out:
@@ -766,7 +774,7 @@ static int perf_file_section__process(struct perf_file_section *self,
 
 	switch (feat) {
 	case HEADER_TRACE_INFO:
-		trace_report(fd);
+		trace_report(fd, false);
 		break;
 
 	case HEADER_BUILD_ID:
@@ -781,12 +789,16 @@ static int perf_file_section__process(struct perf_file_section *self,
 }
 
 static int perf_file_header__read_pipe(struct perf_pipe_file_header *self,
-				       struct perf_header *ph, int fd)
+				       struct perf_header *ph, int fd,
+				       bool repipe)
 {
 	if (do_read(fd, self, sizeof(*self)) <= 0 ||
 	    memcmp(&self->magic, __perf_magic, sizeof(self->magic)))
 		return -1;
 
+	if (repipe && do_write(STDOUT_FILENO, self, sizeof(*self)) < 0)
+		return -1;
+
 	if (self->size != sizeof(*self)) {
 		u64 size = bswap_64(self->size);
 
@@ -804,7 +816,8 @@ static int perf_header__read_pipe(struct perf_session *session, int fd)
 	struct perf_header *self = &session->header;
 	struct perf_pipe_file_header f_header;
 
-	if (perf_file_header__read_pipe(&f_header, self, fd) < 0) {
+	if (perf_file_header__read_pipe(&f_header, self, fd,
+					session->repipe) < 0) {
 		pr_debug("incompatible file format\n");
 		return -EINVAL;
 	}
@@ -1095,12 +1108,17 @@ int event__process_tracing_data(event_t *self,
 	lseek(session->fd, offset + sizeof(struct tracing_data_event),
 	      SEEK_SET);
 
-	size_read = trace_report(session->fd);
+	size_read = trace_report(session->fd, session->repipe);
 
 	padding = ALIGN(size_read, sizeof(u64)) - size_read;
 
 	if (read(session->fd, buf, padding) < 0)
 		die("reading input file");
+	if (session->repipe) {
+		int retw = write(STDOUT_FILENO, buf, padding);
+		if (retw <= 0 || retw != padding)
+			die("repiping tracing data padding");
+	}
 
 	if (size_read + padding != size)
 		die("tracing data size mismatch");
@@ -1109,7 +1127,8 @@ int event__process_tracing_data(event_t *self,
 }
 
 int event__synthesize_build_id(struct dso *pos, u16 misc,
-			       event__handler_t process, struct machine *machine,
+			       event__handler_t process,
+			       struct machine *machine,
 			       struct perf_session *session)
 {
 	event_t ev;
tools/perf/util/session.c +2 -1
index a8dd73e..5d353e7 100644
@@ -77,7 +77,7 @@ int perf_session__create_kernel_maps(struct perf_session *self)
 	return ret;
 }
 
-struct perf_session *perf_session__new(const char *filename, int mode, bool force)
+struct perf_session *perf_session__new(const char *filename, int mode, bool force, bool repipe)
 {
 	size_t len = filename ? strlen(filename) + 1 : 0;
 	struct perf_session *self = zalloc(sizeof(*self) + len);
@@ -97,6 +97,7 @@ struct perf_session *perf_session__new(const char *filename, int mode, bool forc
 	self->cwdlen = 0;
 	self->unknown_events = 0;
 	self->machines = RB_ROOT;
+	self->repipe = repipe;
 	self->ordered_samples.flush_limit = ULLONG_MAX;
 	INIT_LIST_HEAD(&self->ordered_samples.samples_head);
 
tools/perf/util/session.h +2 -1
index 61ca92e..f2b2c6a 100644
@@ -34,6 +34,7 @@ struct perf_session {
 	u64			sample_type;
 	int			fd;
 	bool			fd_pipe;
+	bool			repipe;
 	int			cwdlen;
 	char			*cwd;
 	struct ordered_samples	ordered_samples;
@@ -59,7 +60,7 @@ struct perf_event_ops {
 	bool	ordered_samples;
 };
 
-struct perf_session *perf_session__new(const char *filename, int mode, bool force);
+struct perf_session *perf_session__new(const char *filename, int mode, bool force, bool repipe);
 void perf_session__delete(struct perf_session *self);
 
 void perf_event_header__bswap(struct perf_event_header *self);
tools/perf/util/trace-event-read.c +18 -1
index 44889c9..43f19c1 100644
@@ -51,6 +51,7 @@ static int long_size;
 static unsigned long	page_size;
 
 static ssize_t calc_data_size;
+static bool repipe;
 
 static int do_read(int fd, void *buf, int size)
 {
@@ -62,6 +63,13 @@ static int do_read(int fd, void *buf, int size)
 		if (ret <= 0)
 			return -1;
 
+		if (repipe) {
+			int retw = write(STDOUT_FILENO, buf, ret);
+
+			if (retw <= 0 || retw != ret)
+				die("repiping input file");
+		}
+
 		size -= ret;
 		buf += ret;
 	}
@@ -116,6 +124,13 @@ static char *read_string(void)
 		if (!r)
 			die("no data");
 
+		if (repipe) {
+			int retw = write(STDOUT_FILENO, &c, 1);
+
+			if (retw <= 0 || retw != r)
+				die("repiping input file string");
+		}
+
 		buf[size++] = c;
 
 		if (!c)
@@ -454,7 +469,7 @@ struct record *trace_read_data(int cpu)
 	return data;
 }
 
-ssize_t trace_report(int fd)
+ssize_t trace_report(int fd, bool __repipe)
 {
 	char buf[BUFSIZ];
 	char test[] = { 23, 8, 68 };
@@ -465,6 +480,7 @@ ssize_t trace_report(int fd)
 	ssize_t size;
 
 	calc_data_size = 1;
+	repipe = __repipe;
 
 	input_fd = fd;
 
@@ -499,6 +515,7 @@ ssize_t trace_report(int fd)
 
 	size = calc_data_size - 1;
 	calc_data_size = 0;
+	repipe = false;
 
 	if (show_funcs) {
 		print_funcs();
tools/perf/util/trace-event.h +1 -1
index 1f45d46..ebfee80 100644
@@ -163,7 +163,7 @@ struct record *trace_read_data(int cpu);
 
 void parse_set_info(int nr_cpus, int long_sz);
 
-ssize_t trace_report(int fd);
+ssize_t trace_report(int fd, bool repipe);
 
 void *malloc_or_die(unsigned int size);
 
Signature · 1 line
1.6.4.GIT
#3 lore raw

Re: [RFC][PATCH 2/2] perf: add perf-inject builtin

Em Sat, May 01, 2010 at 01:41:20AM -0500, Tom Zanussi escreveu:

Quoted 16 lines

Currently, perf 'live mode' writes build-ids at the end of the session, which isn't actually useful for processing live mode events.

What would be better would be to have the build-ids sent before any of the samples that reference them, which can be done by processing the event stream and retrieving the build-ids on the first hit. Doing that in perf-record itself, however, is off-limits.

This patch introduces perf-inject, which does the same job while leaving perf-record untouched. Normal mode perf still records the build-ids at the end of the session as it should, but for live mode, perf-inject can be injected in between the record and report steps e.g.:

perf record -o - ./hackbench 10 | perf inject -v -b | perf report -v -i -

Quoted 8 lines

perf-inject reads a perf-record event stream and repipes it to stdout. At any point the processing code can inject other events into the event stream - in this case build-ids (-b option) are read and injected as needed into the event stream.

Build-ids are just the first user of perf-inject - potentially anything that needs userspace processing to augment the trace stream with additional information could make use of this facility.

Good work, I'd just make "read_buildid" be "dso__read_buildid", make its 'self' parameter be a struct dso pointer, first check if had its build id read already, read it if not. Yeah, dsos can change, but that is also not supported in 'perf record', where we do it at the end.

I.e. reducing the cost of build-id processing even in live mode is also interesting and OK for the normal case of not changing DSOs while a session is running.

- Arnaldo