Skip to contentSkip to conversation

perf: 'live mode'

RFC PATCH Superseded Tom Zanussi 4 participants 17 messages Mar 3, 2010 – Mar 28, 2010 perf events

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover perf: 'live mode'

Currently, a perf session entails two steps: first 'perf record' or 'perf trace record' records the perf data to disk, then 'perf report' or 'perf trace report' reads the saved data from disk and reports the results.

This experimental patchset makes some changes to perf that instead allow the perf data to be piped directly from the record step to the report step, without ever touching the disk.

This is especially useful for 'perf trace' - adding this capability means that the trace scripts are no longer relegated to simple post-processing, but can be run in a continuous 'live mode', forever processing the event stream and e.g. periodically dumping current results, essentially becoming special-purpose 'top' applications, or continuously scanning the event stream for arbitrarily complex conditions to flag, etc...

Being able to feed the event stream over a pipe also makes it possible to do things like trace over the network using e.g. netcat.

It turns out that perf is pretty close to being able to do this already, with the exception of the header data; most of the work of this patchset deals with changing that. It does so in a mainly additive way: it doesn't make any changes to the existing disk format or normal disk-mode processing, just adds special treatment for the cases when 'perf [trace] record' records to stdout or 'perf [trace] report reads from stdin.

Here are some quick examples. Basically using '-' as the filename to the -o and -i options send/read the data from stdout/stdin:

Here's a simple 'perf record' | 'perf report' run:

root@tropicana:~# perf record -o - -c 1 -f -a -M -R -e raw_syscalls:sys_enter | perf report -i -
^C# Samples: 120234
#
# Overhead          Command              Shared Object  Symbol
# ........  ...............  .........................  ......
#
    98.65%             perf  libpthread-2.8.90.so       [.] __read
     0.46%             perf  libpthread-2.8.90.so       [.] __write_nocancel
     0.32%             perf  libpthread-2.8.90.so       [.] __open_nocancel
     0.29%             perf  libpthread-2.8.90.so       [.] __libc_close
     0.07%             perf  libc-2.8.90.so             [.] 0x000000000a4b15
     0.03%             perf  libc-2.8.90.so             [.] __fxstat64
     0.03%             perf  libc-2.8.90.so             [.] __open
     0.03%             perf  libc-2.8.90.so             [.] close
     0.02%             Xorg  libc-2.8.90.so             [.] __select
     0.01%             Xorg  libpthread-2.8.90.so       [.] __read
     0.01%             perf  libpthread-2.8.90.so       [.] __libc_lseek
     0.01%             Xorg  libc-2.8.90.so             [.] setitimer
     0.01%             perf  libc-2.8.90.so             [.] __xstat64
     0.01%          firefox  libpthread-2.8.90.so       [.] __read
     0.00%             perf  libc-2.8.90.so             [.] llseek
     0.00%             Xorg  libpthread-2.8.90.so       [.] __restore_rt
     0.00%             Xorg  libc-2.8.90.so             [.] sigprocmask
     0.00%             Xorg  libc-2.8.90.so             [.] __poll
     0.00%            wterm  libc-2.8.90.so             [.] __read
     0.00%  gnome-screensav  libpthread-2.8.90.so       [.] __read
     0.00%          firefox  libc-2.8.90.so             [.] __poll
     0.00%             Xorg  libc-2.8.90.so             [.] writev
     0.00%  gnome-settings-  libpthread-2.8.90.so       [.] __read
     0.00%            wterm  libc-2.8.90.so             [.] __select
     0.00%             perf  libc-2.8.90.so             [.] statfs
     0.00%           mysqld  libc-2.8.90.so             [.] __select
     0.00%          firefox  libpthread-2.8.90.so       [.] __pthread_cond_timedwait
     0.00%          firefox  libpthread-2.8.90.so       [.] __lll_unlock_wake
     0.00%          firefox  libpthread-2.8.90.so       [.] 0x0000000000decb
     0.00%          firefox  libc-2.8.90.so             [.] __select
     0.00%             perf  libc-2.8.90.so             [.] getcwd
     0.00%             perf  libc-2.8.90.so             [.] mmap64
     0.00%  gnome-settings-  libc-2.8.90.so             [.] __poll
     0.00%  gnome-screensav  libc-2.8.90.so             [.] __poll
     0.00%          firefox  libc-2.8.90.so             [.] writev
     0.00%          apache2  libpthread-2.8.90.so       [.] __waitpid
     0.00%          apache2  libc-2.8.90.so             [.] __select
#
# (For a higher level overview, try: perf report --sort comm,dso)
#

Included in this patchset are a couple of 'top' scripts, rwtop and sctop, that are essentially just modifications of existing scripts. Basically the original scripts were modified to add a 5 second timer. In the handler for the timer, the current output summary is printed, and the state is cleared and begun anew, ad infinitum.

Here are the new scripts as shown in the perf trace list:

root@tropicana:~# perf trace -l
List of available trace scripts:
  workqueue-stats                      workqueue stats (ins/exe/create/destroy)
  wakeup-latency                       system-wide min/max/avg wakeup latency
  rw-by-file <comm>                    r/w activity for a program, by file
  rwtop                                system-wide r/w top
  failed-syscalls [comm]               system-wide failed syscalls
  rw-by-pid                            system-wide r/w activity
  syscall-counts-by-pid [comm]         system-wide syscall counts, by pid
  failed-syscalls-by-pid [comm]        system-wide failed syscalls, by pid
  sctop [comm]                         syscall top
  syscall-counts [comm]                system-wide syscall counts

And here's a few iterations of the output of the 'sctop' Python script:

root@tropicana:~# perf trace record sctop | perf trace report sctop
perf trace started with Python script /root/libexec/perf-core/scripts/python/sctop.py

syscall events:

event count
---------------------------------------- -----------
2 179
3 158

0                                                 79
78                                                24
5                                                 13
1                                                  3
16                                                 2
298                                                1
137                                                1
79                                                 1
72                                                 1
8                                                  1

syscall events:

event count
---------------------------------------- -----------
0 659287
202 1995

1                                                812
2                                                246
3                                                224
78                                                56
12                                                51
5                                                 23
23                                                22
8                                                 18
7                                                 18
4                                                  9
38                                                 6
61                                                 5
254                                                4
20                                                 2
16                                                 2
137                                                1
15                                                 1
14                                                 1
9                                                  1

syscall events:

event count
---------------------------------------- -----------
0 647178
202 1209
1 324
12 52
7 29

14                                                 6
16                                                 4
23                                                 3
3                                                  3
72                                                 2
21                                                 1
11                                                 1
9                                                  1
5                                                  1
4                                                  1
2                                                  1

.
.
.

Obviously, it would be better to have a real top-like display for these rather than a continuously scrolling mode like this, and of course it will be much more useful once we get the syscall name injection events going (the column on the left shows syscall numbers only).

Last, but not least, here's an example that uses netcat to run the 'rwtop' Perl script over the network:

On the system being traced:

root@tropicana:~# perf trace record rwtop | nc localhost 7777
^C

On the system collecting the data and printing the output every 5 seconds:

root@tropicana:~# nc -l -p 7777 | perf trace report rwtop
perf trace started with Perl script /root/libexec/perf-core/scripts/perl/rwtop.pl

read counts by pid:

pid comm # reads bytes_requested bytes_read

------  --------------------  -----------  ----------  ----------
  6672  perf                      156016     6238975     6238980
  6670  nc                           741     6070272     6054460
  6675  perf                        3235     9445712     4589634
  6674  nc                          1376    11272192     1586892
  6342  gvfsd-trash                    2        4096         194
  5856  Xorg                           5        8264          76
  6335  trashapplet                    5       16384          48
  6378  gnome-power-man                3       12288          32
  6298  gnome-panel                    1        4096           0

write counts by pid:

pid comm # writes bytes_written

------  --------------------  -----------  ----------
  6675  perf                       15651    16412612
  6670  nc                           741     6054460
  6674  nc                          1376     1586892
  6342  gvfsd-trash                    2          93
  6676  gvfsd-trash                    1          35

read counts by pid:

pid comm # reads bytes_requested bytes_read

------  --------------------  -----------  ----------  ----------
  6672  perf                      166458     6657104     6656232
  6670  nc                           787     6447104     6447104
  6342  gvfsd-trash                    2        4096         194
  6335  trashapplet                    5       16384          48

write counts by pid:

pid comm # writes bytes_written

------  --------------------  -----------  ----------
  6675  perf                       13734    15206704
  6670  nc                           787     6447104
  6342  gvfsd-trash                    2          93
  6677  gvfsd-trash                    1          35

read counts by pid:

pid comm # reads bytes_requested bytes_read

------  --------------------  -----------  ----------  ----------
  6672  perf                      167645     6704272     6704048
  6670  nc                           793     6496256     6529024
  6298  gnome-panel                    1        4096           0

write counts by pid:

pid comm # writes bytes_written

------  --------------------  -----------  ----------
  6675  perf                       13473    15011968
  6670  nc                           797     6529024

failed to read event payload

read counts by pid:

pid comm # reads bytes_requested bytes_read

------  --------------------  -----------  ----------  ----------
  6670  nc                           217     1777664     1875968
  6672  perf                       45782     1830816     1830560
  6451  wterm                          3       19873         607
  5856  Xorg                           3          72          72
  6480  wterm                          3       12288          32
  6263  gnome-settings-                3       12288          32
  6326  gnome-screensav                4       16384          32

write counts by pid:

pid comm # writes bytes_written

------  --------------------  -----------  ----------
  6675  perf                        3821     4635048
  6670  nc                           229     1875968
  6672  perf                          11         592

perf trace Perl script stopped

There are some rough edges and inefficiencies, and I guess 'event injection' may be a better way to do this in the end, but it seems to work pretty well already, and I think it shows how useful such a capability can be.

Tom Zanussi (7):
perf: introduce special handling for pipe input/output
perf: add pipe-specific header read/write and event processing code
perf: convert perf header attrs into attr events
perf: convert perf event types into event type events
perf: convert perf tracing data into a tracing_data event
perf: convert perf header build_ids into build_id events perf trace/scripting: rwtop and sctop scripts

 tools/perf/builtin-record.c                |   66 +++++-
 tools/perf/builtin-report.c                |   13 +-
 tools/perf/builtin-trace.c                 |   11 +
 tools/perf/scripts/perl/bin/rwtop-record   |    2 +
 tools/perf/scripts/perl/bin/rwtop-report   |    6 +
 tools/perf/scripts/perl/rwtop.pl           |  165 +++++++++++++
 tools/perf/scripts/python/bin/sctop-record |    2 +
 tools/perf/scripts/python/bin/sctop-report |    4 +
 tools/perf/scripts/python/sctop.py         |   64 +++++
 tools/perf/util/event.h                    |   35 +++
 tools/perf/util/header.c                   |  348 +++++++++++++++++++++++++++-
 tools/perf/util/header.h                   |   38 +++-
 tools/perf/util/session.c                  |  214 ++++++++++++++++-
 tools/perf/util/session.h                  |   23 ++-
 tools/perf/util/trace-event-info.c         |   24 ++
 tools/perf/util/trace-event-read.c         |   89 ++++----
 tools/perf/util/trace-event.h              |    4 +-
 17 files changed, 1035 insertions(+), 73 deletions(-)
 create mode 100644 tools/perf/scripts/perl/bin/rwtop-record
 create mode 100644 tools/perf/scripts/perl/bin/rwtop-report
 create mode 100644 tools/perf/scripts/perl/rwtop.pl
 create mode 100644 tools/perf/scripts/python/bin/sctop-record
 create mode 100644 tools/perf/scripts/python/bin/sctop-report
 create mode 100644 tools/perf/scripts/python/sctop.py
#1 lore raw

RFC PATCH 1/7 perf: introduce special handling for pipe input/output

Adds special treatment for stdin/stdout - if the user specifies '-o -' to perf record or '-i -' to perf report et al, the intent is that the event stream be written to stdout or read from stdin rather than a disk file.

This initial patch just handles parsing the cmdline for those modes and setting a couple variables to be used by the code in later patches.

---
 tools/perf/builtin-record.c |   10 ++++++++--
 tools/perf/builtin-report.c |    3 ++-
 tools/perf/util/session.c   |   10 ++++++++++
 tools/perf/util/session.h   |    1 +
 4 files changed, 21 insertions(+), 3 deletions(-)
tools/perf/builtin-record.c +8 -2
index 771533c..4fe87e3 100644
@@ -35,6 +35,7 @@ static unsigned int		page_size;
 static unsigned int		mmap_pages			=    128;
 static int			freq				=   1000;
 static int			output;
+static int			pipe_output			=      0;
 static const char		*output_name			= "perf.data";
 static int			group				=      0;
 static unsigned int		realtime_prio			=      0;
@@ -431,7 +432,9 @@ static int __cmd_record(int argc, const char **argv)
 		exit(-1);
 	}
 
-	if (!stat(output_name, &st) && st.st_size) {
+	if (!strcmp(output_name, "-"))
+		pipe_output = 1;
+	else if (!stat(output_name, &st) && st.st_size) {
 		if (!force) {
 			if (!append_file) {
 				pr_err("Error, output file %s exists, use -A "
@@ -456,7 +459,10 @@ static int __cmd_record(int argc, const char **argv)
 	else
 		flags |= O_TRUNC;
 
-	output = open(output_name, flags, S_IRUSR|S_IWUSR);
+	if (pipe_output)
+		output = STDOUT_FILENO;
+	else
+		output = open(output_name, flags, S_IRUSR | S_IWUSR);
 	if (output < 0) {
 		perror("failed to create output file");
 		exit(-1);
tools/perf/builtin-report.c +2 -1
index cfc655d..ca12efa 100644
@@ -359,7 +359,8 @@ int cmd_report(int argc, const char **argv, const char *prefix __used)
 {
 	argc = parse_options(argc, argv, options, report_usage, 0);
 
-	setup_pager();
+	if (strcmp(input_name, "-") != 0)
+		setup_pager();
 
 	if (symbol__init() < 0)
 		return -1;
tools/perf/util/session.c +10 -0
index 0de7258..dd1e923 100644
@@ -14,6 +14,16 @@ static int perf_session__open(struct perf_session *self, bool force)
 {
 	struct stat input_stat;
 
+	if (!strcmp(self->filename, "-")) {
+		self->fd_pipe = true;
+		self->fd = STDIN_FILENO;
+
+		if (perf_header__read(&self->header, self->fd) < 0)
+			pr_err("incompatible file format");
+
+		return 0;
+	}
+
 	self->fd = open(self->filename, O_RDONLY);
 	if (self->fd < 0) {
 		pr_err("failed to open file: %s", self->filename);
tools/perf/util/session.h +1 -0
index 31950fc..dbfb74a 100644
@@ -26,6 +26,7 @@ struct perf_session {
 	u64			sample_type;
 	struct ref_reloc_sym	ref_reloc_sym;
 	int			fd;
+	bool			fd_pipe;
 	int			cwdlen;
 	char			*cwd;
 	char filename[0];
Signature · 1 line
1.6.4.GIT
#2 lore raw

RFC PATCH 2/7 perf: add pipe-specific header read/write and event processing code

In reply to Tom Zanussi

This patch makes several changes to allow the perf event stream to be sent and received over a pipe:

- adds pipe-specific versions of the header read/write code

- adds pipe-specific version of the event processing code

- adds a range of event types to be used for header or other pseudo events, above the range used by the kernel

- adds code to interpret return values >0 from event processing functions as 'actions'. The first action defined is for stopping the event-processing loop at the first non-header event. So initially there's still a restriction that header events come first in the event stream.

Note that none of this affects the existing perf data file format or processing - this code only comes into play if perf output is sent to stdout (or is read from stdin).

---
 tools/perf/builtin-record.c |    8 ++-
 tools/perf/builtin-report.c |   10 +++
 tools/perf/builtin-trace.c  |   11 +++
 tools/perf/util/event.h     |    4 +
 tools/perf/util/header.c    |   62 ++++++++++++++++++-
 tools/perf/util/header.h    |    8 ++-
 tools/perf/util/session.c   |  145 +++++++++++++++++++++++++++++++++++++++---
 tools/perf/util/session.h   |   16 +++++
 8 files changed, 249 insertions(+), 15 deletions(-)
tools/perf/builtin-record.c +6 -2
index 4fe87e3..b676689 100644
@@ -475,7 +475,7 @@ static int __cmd_record(int argc, const char **argv)
 	}
 
 	if (!file_new) {
-		err = perf_header__read(&session->header, output);
+		err = perf_header__read(session, output);
 		if (err < 0)
 			return err;
 	}
@@ -554,7 +554,11 @@ static int __cmd_record(int argc, const char **argv)
 			open_counters(i, target_pid);
 	}
 
-	if (file_new) {
+	if (pipe_output) {
+		err = perf_header__write_pipe(&session->header, output);
+		if (err < 0)
+			return err;
+	} else if (file_new) {
 		err = perf_header__write(&session->header, output, false);
 		if (err < 0)
 			return err;
tools/perf/builtin-report.c +10 -0
index ca12efa..603f705 100644
@@ -193,11 +193,21 @@ static struct perf_event_ops event_ops = {
 	.read	= process_read_event,
 };
 
+extern volatile int done;
+
+static void sig_handler(int sig __attribute__((__unused__)))
+{
+	done = 1;
+}
+
 static int __cmd_report(void)
 {
 	int ret = -EINVAL;
 	struct perf_session *session;
 
+	signal(SIGCHLD, sig_handler);
+	signal(SIGINT, sig_handler);
+
 	session = perf_session__new(input_name, O_RDONLY, force);
 	if (session == NULL)
 		return -ENOMEM;
tools/perf/builtin-trace.c +11 -0
index 407041d..2a56dda 100644
@@ -106,8 +106,18 @@ static struct perf_event_ops event_ops = {
 	.comm	= event__process_comm,
 };
 
+extern volatile int done;
+
+static void sig_handler(int sig __unused)
+{
+	done = 1;
+}
+
 static int __cmd_trace(struct perf_session *session)
 {
+	signal(SIGCHLD, sig_handler);
+	signal(SIGINT, sig_handler);
+
 	return perf_session__process_events(session, &event_ops);
 }
 
@@ -609,6 +619,7 @@ int cmd_trace(int argc, const char **argv, const char *prefix __used)
 			return -1;
 		}
 
+		perf_header__read(session, input);
 		err = scripting_ops->generate_script("perf-trace");
 		goto out;
 	}
tools/perf/util/event.h +4 -0
index 50a7132..cec3067 100644
@@ -83,6 +83,10 @@ struct build_id_event {
 	char			 filename[];
 };
 
+enum perf_header_event_type { /* above any possible kernel type */
+	PERF_RECORD_HEADER_MAX			= 64,
+};
+
 typedef union event_union {
 	struct perf_event_header	header;
 	struct ip_event			ip;
tools/perf/util/header.c +61 -1

80 lines not on this page · raw message

tools/perf/util/header.h +7 -1
index 82a6af7..714b1d5 100644
@@ -39,6 +39,11 @@ struct perf_file_header {
 	DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
 };
 
+struct perf_file_header_pipe {
+	u64				magic;
+	u64				size;
+};
+
 struct perf_header;
 
 int perf_file_header__read(struct perf_file_header *self,
@@ -60,8 +65,9 @@ struct perf_header {
 int perf_header__init(struct perf_header *self);
 void perf_header__exit(struct perf_header *self);
 
-int perf_header__read(struct perf_header *self, int fd);
+int perf_header__read(struct perf_session *session, int fd);
 int perf_header__write(struct perf_header *self, int fd, bool at_exit);
+int perf_header__write_pipe(struct perf_header *self, int fd);
 
 int perf_header__add_attr(struct perf_header *self,
 			  struct perf_header_attr *attr);
tools/perf/util/session.c +134 -11

215 lines not on this page · raw message

tools/perf/util/session.h +16 -0
index dbfb74a..ab1fcbf 100644
@@ -46,6 +46,18 @@ struct perf_event_ops {
 		 unthrottle;
 };
 
+/*
+ * interpret process_event return values as actions.
+ * PERF_PROCESS_EVENT_MAX or greater is interpreted as 'skip ahead
+ * that many bytes'.  Since that can never be less than
+ * sizeof(perf_event_header), 1 through sizeof(perf_event_header) can
+ * be interpreted as 'actions'.
+*/
+enum perf_process_event_action {
+	PERF_PROCESS_EVENT_STOP			= 1,
+	PERF_PROCESS_EVENT_MAX
+};
+
 struct perf_session *perf_session__new(const char *filename, int mode, bool force);
 void perf_session__delete(struct perf_session *self);
 
@@ -86,4 +98,8 @@ static inline struct map *
 {
 	return map_groups__new_module(&self->kmaps, start, filename);
 }
+
+void perf_session__update_sample_type(struct perf_session *self);
+void perf_event_ops__fill_stop(struct perf_event_ops *handler);
+
 #endif /* __PERF_SESSION_H */
Signature · 1 line
1.6.4.GIT
#3 lore raw

Re: [RFC PATCH 2/7] perf: add pipe-specific header read/write and event processing code

Quoted 10 lines · Tom Zanussi
tools/perf/util/header.c +9 -0
+int perf_header__write_pipe(struct perf_header *self, int fd)
+{
+	struct perf_file_header_pipe f_header;
+	int err;
+
+	f_header = (struct perf_file_header_pipe){
+		.magic	   = PERF_MAGIC,
+		.size	   = sizeof(f_header),
+	};

tools/perf/util/header.c out_free: · line 448

Quoted 10 lines
tools/perf/util/header.c +10 -0
+
+	err = do_write(fd, &f_header, sizeof(f_header));
+	if (err < 0) {
+		pr_debug("failed to write perf pipe header\n");
+		return err;
+	}
+
+	self->frozen = 1;
+	return 0;
+}

I actually wonder why you bother with a header in the pipe mode, since it only contains MAGIC and its size.

#4 lore raw

Re: [RFC PATCH 2/7] perf: add pipe-specific header read/write and event processing code

Em Sat, Mar 27, 2010 at 04:14:57AM +0100, Frederic Weisbecker escreveu:

Quoted 14 lines · Tom Zanussi

tools/perf/util/header.c out_free: · line 438

Quoted 10 lines
tools/perf/util/header.c +9 -0
+int perf_header__write_pipe(struct perf_header *self, int fd)
+{
+	struct perf_file_header_pipe f_header;
+	int err;
+
+	f_header = (struct perf_file_header_pipe){
+		.magic	   = PERF_MAGIC,
+		.size	   = sizeof(f_header),
+	};

I actually wonder why you bother with a header in the pipe mode, since it only contains MAGIC and its size.

Because he wants to detect endianness :-)

- ARnaldo

#5 lore raw

Re: [RFC PATCH 2/7] perf: add pipe-specific header read/write and event processing code

Em Sat, Mar 27, 2010 at 07:57:47PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 16 lines

Em Sat, Mar 27, 2010 at 04:14:57AM +0100, Frederic Weisbecker escreveu:

Quoted 14 lines · Tom Zanussi

tools/perf/util/header.c out_free: · line 438

Quoted 10 lines
tools/perf/util/header.c +9 -0
+int perf_header__write_pipe(struct perf_header *self, int fd)
+{
+	struct perf_file_header_pipe f_header;
+	int err;
+
+	f_header = (struct perf_file_header_pipe){
+		.magic	   = PERF_MAGIC,
+		.size	   = sizeof(f_header),
+	};

I actually wonder why you bother with a header in the pipe mode, since it only contains MAGIC and its size.

Quoted 1 line

Because he wants to detect endianness :-)

Elaborating: piping over netcat from a big endian to a little endian machine works because he used the ->needs_swap logic I implemented to allow cross platform analysis.

Its just a matter of having the DSOS with matching build-ids in the cache of the machine running 'perf report'.

- Arnaldo

#6 lore raw

Re: [RFC PATCH 2/7] perf: add pipe-specific header read/write and event processing code

Quoted 19 lines · Arnaldo Carvalho de Melo

Em Sat, Mar 27, 2010 at 07:57:47PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 16 lines

Em Sat, Mar 27, 2010 at 04:14:57AM +0100, Frederic Weisbecker escreveu:

Quoted 14 lines · Tom Zanussi

tools/perf/util/header.c out_free: · line 438

Quoted 10 lines
tools/perf/util/header.c +9 -0
+int perf_header__write_pipe(struct perf_header *self, int fd)
+{
+	struct perf_file_header_pipe f_header;
+	int err;
+
+	f_header = (struct perf_file_header_pipe){
+		.magic	   = PERF_MAGIC,
+		.size	   = sizeof(f_header),
+	};

I actually wonder why you bother with a header in the pipe mode, since it only contains MAGIC and its size.

Quoted 1 line

Because he wants to detect endianness :-)

Quoted 8 lines

Elaborating: piping over netcat from a big endian to a little endian machine works because he used the ->needs_swap logic I implemented to allow cross platform analysis.

Its just a matter of having the DSOS with matching build-ids in the cache of the machine running 'perf report'.

- Arnaldo

Ok, thanks for the explanations :-)

#7 lore raw

Re: [RFC PATCH 2/7] perf: add pipe-specific header read/write and event processing code

Em Sun, Mar 28, 2010 at 12:12:05AM +0100, Frederic Weisbecker escreveu:

Quoted 8 lines · Arnaldo Carvalho de Melo
Quoted 7 lines

Em Sat, Mar 27, 2010 at 07:57:47PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 5 lines

Em Sat, Mar 27, 2010 at 04:14:57AM +0100, Frederic Weisbecker escreveu:

Quoted 2 lines

I actually wonder why you bother with a header in the pipe mode, since it only contains MAGIC and its size.

Because he wants to detect endianness :-)

Quoted 8 lines
Quoted 6 lines

Elaborating: piping over netcat from a big endian to a little endian machine works because he used the ->needs_swap logic I implemented to allow cross platform analysis.

Its just a matter of having the DSOS with matching build-ids in the cache of the machine running 'perf report'.

Ok, thanks for the explanations :-)

Humm, now that I read this again, the build-ids now are in the headers, which will not be pushed as it is the last step in 'perf record', after we stop recording events, so only if we make sure that the binaries are the same is that we will be able to make sense of it over the network.

One more reason for the loader of DSOs to read the pre calculated build-id ELF session to tell the kernel the 20 bytes build id that in turn will inject it in the PERF_RECORD_MMAP event.

- Arnaldo

#8 lore raw

RFC PATCH 3/7 perf: convert perf header attrs into attr events

In reply to Tom Zanussi

Bypasses the attr perf header code and replaces it with a synthesized event and processing function that accomplishes the same thing, used when reading/writing perf data to/from a pipe.

Making the attrs into events allows them to be streamed over a pipe along with the rest of the header data (in later patches). It also paves the way to allowing events to be added and removed from perf sessions dynamically.

---
 tools/perf/builtin-record.c |   10 +++++
 tools/perf/util/event.h     |   10 +++++-
 tools/perf/util/header.c    |   80 +++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/header.h    |    8 ++++
 tools/perf/util/session.c   |   27 ++++++++++++++
 tools/perf/util/session.h   |    3 +-
 6 files changed, 136 insertions(+), 2 deletions(-)
tools/perf/builtin-record.c +10 -0
index b676689..3ba466f 100644
@@ -566,6 +566,16 @@ static int __cmd_record(int argc, const char **argv)
 
 	post_processing_offset = lseek(output, 0, SEEK_CUR);
 
+	if (pipe_output) {
+		err = event__synthesize_attrs(&session->header,
+					      process_synthesized_event,
+					      session);
+		if (err < 0) {
+			pr_err("Couldn't synthesize attrs.\n");
+			return err;
+		}
+	}
+
 	err = event__synthesize_kernel_mmap(process_synthesized_event,
 					    session, "_text");
 	if (err < 0) {
tools/perf/util/event.h +9 -1
index cec3067..83e5d08 100644
@@ -84,7 +84,14 @@ struct build_id_event {
 };
 
 enum perf_header_event_type { /* above any possible kernel type */
-	PERF_RECORD_HEADER_MAX			= 64,
+	PERF_RECORD_HEADER_ATTR			= 64,
+	PERF_RECORD_HEADER_MAX
+};
+
+struct attr_event {
+	struct perf_event_header header;
+	struct perf_event_attr attr;
+	u64 id[];
 };
 
 typedef union event_union {
@@ -96,6 +103,7 @@ typedef union event_union {
 	struct lost_event		lost;
 	struct read_event		read;
 	struct sample_event		sample;
+	struct attr_event		attr;
 } event_t;
 
 struct events_stats {
tools/perf/util/header.c +80 -0

89 lines not on this page · raw message

tools/perf/util/header.h +8 -0
index 714b1d5..f5fd82c 100644
@@ -95,4 +95,12 @@ int build_id_cache__add_s(const char *sbuild_id, const char *debugdir,
 			  const char *name, bool is_kallsyms);
 int build_id_cache__remove_s(const char *sbuild_id, const char *debugdir);
 
+int event__synthesize_attr(struct perf_event_attr *attr, u16 ids, u64 *id,
+			   event__handler_t process,
+			   struct perf_session *session);
+int event__synthesize_attrs(struct perf_header *self,
+			    event__handler_t process,
+			    struct perf_session *session);
+int event__process_attr(event_t *self, struct perf_session *session);
+
 #endif /* __PERF_HEADER_H */
tools/perf/util/session.c +27 -0
index 3a3db33..ff51105 100644
@@ -209,6 +209,8 @@ static void perf_event_ops__fill_defaults(struct perf_event_ops *handler)
 		handler->throttle = process_event_stub;
 	if (handler->unthrottle == NULL)
 		handler->unthrottle = process_event_stub;
+	if (handler->attr == NULL)
+		handler->attr = process_event_stub;
 }
 
 static int process_event_stop(event_t *event __used,
@@ -229,6 +231,7 @@ void perf_event_ops__fill_stop(struct perf_event_ops *handler)
 	handler->read = process_event_stop;
 	handler->throttle = process_event_stop;
 	handler->unthrottle = process_event_stop;
+	handler->attr = process_event_stop;
 }
 
 static const char *event__name[] = {
@@ -242,6 +245,7 @@ static const char *event__name[] = {
 	[PERF_RECORD_FORK]	 = "FORK",
 	[PERF_RECORD_READ]	 = "READ",
 	[PERF_RECORD_SAMPLE]	 = "SAMPLE",
+	[PERF_RECORD_HEADER_ATTR]	 = "ATTR",
 };
 
 unsigned long event__total[PERF_RECORD_HEADER_MAX];
@@ -308,6 +312,26 @@ static void event__read_swap(event_t *self)
 	self->read.id		= bswap_64(self->read.id);
 }
 
+static void event__attr_swap(event_t *self)
+{
+	size_t size;
+
+	self->attr.attr.type		= bswap_32(self->attr.attr.type);
+	self->attr.attr.size		= bswap_32(self->attr.attr.size);
+	self->attr.attr.config		= bswap_64(self->attr.attr.config);
+	self->attr.attr.sample_period	= bswap_64(self->attr.attr.sample_period);
+	self->attr.attr.sample_type	= bswap_64(self->attr.attr.sample_type);
+	self->attr.attr.read_format	= bswap_64(self->attr.attr.read_format);
+	self->attr.attr.wakeup_events	= bswap_32(self->attr.attr.wakeup_events);
+	self->attr.attr.bp_type		= bswap_32(self->attr.attr.bp_type);
+	self->attr.attr.bp_addr		= bswap_64(self->attr.attr.bp_addr);
+	self->attr.attr.bp_len		= bswap_64(self->attr.attr.bp_len);
+
+	size = self->header.size;
+	size -= (void *)&self->attr.id - (void *)self;
+	mem_bswap_64(self->attr.id, size);
+}
+
 typedef void (*event__swap_op)(event_t *self);
 
 static event__swap_op event__swap_ops[] = {
@@ -318,6 +342,7 @@ static event__swap_op event__swap_ops[] = {
 	[PERF_RECORD_LOST]   = event__all64_swap,
 	[PERF_RECORD_READ]   = event__read_swap,
 	[PERF_RECORD_SAMPLE] = event__all64_swap,
+	[PERF_RECORD_HEADER_ATTR]   = event__attr_swap,
 	[PERF_RECORD_HEADER_MAX]    = NULL,
 };
 
@@ -358,6 +383,8 @@ static int perf_session__process_event(struct perf_session *self,
 		return ops->throttle(event, self);
 	case PERF_RECORD_UNTHROTTLE:
 		return ops->unthrottle(event, self);
+	case PERF_RECORD_HEADER_ATTR:
+		return ops->attr(event, self);
 	default:
 		self->unknown_events++;
 		return -1;
tools/perf/util/session.h +2 -1
index ab1fcbf..7365a24 100644
@@ -43,7 +43,8 @@ struct perf_event_ops {
 		 lost,
 		 read,
 		 throttle,
-		 unthrottle;
+		 unthrottle,
+		 attr;
 };
 
 /*
Signature · 1 line
1.6.4.GIT
#9 lore raw

RFC PATCH 4/7 perf: convert perf event types into event type events

In reply to Tom Zanussi

Bypasses the event type perf header code and replaces it with a synthesized event and processing function that accomplishes the same thing, used when reading/writing perf data to/from a pipe.

---
 tools/perf/builtin-record.c |    7 +++++
 tools/perf/util/event.h     |   14 +++++++++
 tools/perf/util/header.c    |   63 ++++++++++++++++++++++++++++++++++++++-----
 tools/perf/util/header.h    |    9 ++++++
 tools/perf/util/session.c   |   13 +++++++++
 tools/perf/util/session.h   |    3 +-
 6 files changed, 101 insertions(+), 8 deletions(-)
tools/perf/builtin-record.c +7 -0
index 3ba466f..0e3d4cd 100644
@@ -574,6 +574,13 @@ static int __cmd_record(int argc, const char **argv)
 			pr_err("Couldn't synthesize attrs.\n");
 			return err;
 		}
+
+		err = event__synthesize_event_types(process_synthesized_event,
+						    session);
+		if (err < 0) {
+			pr_err("Couldn't synthesize event_types.\n");
+			return err;
+		}
 	}
 
 	err = event__synthesize_kernel_mmap(process_synthesized_event,
tools/perf/util/event.h +14 -0
index 83e5d08..dc6d27b 100644
@@ -85,6 +85,7 @@ struct build_id_event {
 
 enum perf_header_event_type { /* above any possible kernel type */
 	PERF_RECORD_HEADER_ATTR			= 64,
+	PERF_RECORD_HEADER_EVENT_TYPE		= 65,
 	PERF_RECORD_HEADER_MAX
 };
 
@@ -94,6 +95,18 @@ struct attr_event {
 	u64 id[];
 };
 
+#define MAX_EVENT_NAME 64
+
+struct perf_trace_event_type {
+	u64	event_id;
+	char	name[MAX_EVENT_NAME];
+};
+
+struct event_type_event {
+	struct perf_event_header header;
+	struct perf_trace_event_type event_type;
+};
+
 typedef union event_union {
 	struct perf_event_header	header;
 	struct ip_event			ip;
@@ -104,6 +117,7 @@ typedef union event_union {
 	struct read_event		read;
 	struct sample_event		sample;
 	struct attr_event		attr;
+	struct event_type_event		event_type;
 } event_t;
 
 struct events_stats {
tools/perf/util/header.c +56 -7

78 lines not on this page · raw message

tools/perf/util/header.h +9 -0
index f5fd82c..9d18c9b 100644
@@ -103,4 +103,13 @@ int event__synthesize_attrs(struct perf_header *self,
 			    struct perf_session *session);
 int event__process_attr(event_t *self, struct perf_session *session);
 
+int event__synthesize_event_type(u64 event_id, char *name,
+				 event__handler_t process,
+				 struct perf_session *session);
+int event__synthesize_event_types(event__handler_t process,
+				  struct perf_session *session);
+int event__process_event_type(event_t *self,
+			      struct perf_session *session);
+
+
 #endif /* __PERF_HEADER_H */
tools/perf/util/session.c +13 -0
index ff51105..d704bdf 100644
@@ -211,6 +211,8 @@ static void perf_event_ops__fill_defaults(struct perf_event_ops *handler)
 		handler->unthrottle = process_event_stub;
 	if (handler->attr == NULL)
 		handler->attr = process_event_stub;
+	if (handler->event_type == NULL)
+		handler->event_type = process_event_stub;
 }
 
 static int process_event_stop(event_t *event __used,
@@ -232,6 +234,7 @@ void perf_event_ops__fill_stop(struct perf_event_ops *handler)
 	handler->throttle = process_event_stop;
 	handler->unthrottle = process_event_stop;
 	handler->attr = process_event_stop;
+	handler->event_type = process_event_stop;
 }
 
 static const char *event__name[] = {
@@ -246,6 +249,7 @@ static const char *event__name[] = {
 	[PERF_RECORD_READ]	 = "READ",
 	[PERF_RECORD_SAMPLE]	 = "SAMPLE",
 	[PERF_RECORD_HEADER_ATTR]	 = "ATTR",
+	[PERF_RECORD_HEADER_EVENT_TYPE]	 = "EVENT_TYPE",
 };
 
 unsigned long event__total[PERF_RECORD_HEADER_MAX];
@@ -332,6 +336,12 @@ static void event__attr_swap(event_t *self)
 	mem_bswap_64(self->attr.id, size);
 }
 
+static void event__event_type_swap(event_t *self)
+{
+	self->event_type.event_type.event_id =
+		bswap_64(self->event_type.event_type.event_id);
+}
+
 typedef void (*event__swap_op)(event_t *self);
 
 static event__swap_op event__swap_ops[] = {
@@ -343,6 +353,7 @@ static event__swap_op event__swap_ops[] = {
 	[PERF_RECORD_READ]   = event__read_swap,
 	[PERF_RECORD_SAMPLE] = event__all64_swap,
 	[PERF_RECORD_HEADER_ATTR]   = event__attr_swap,
+	[PERF_RECORD_HEADER_EVENT_TYPE]   = event__event_type_swap,
 	[PERF_RECORD_HEADER_MAX]    = NULL,
 };
 
@@ -385,6 +396,8 @@ static int perf_session__process_event(struct perf_session *self,
 		return ops->unthrottle(event, self);
 	case PERF_RECORD_HEADER_ATTR:
 		return ops->attr(event, self);
+	case PERF_RECORD_HEADER_EVENT_TYPE:
+		return ops->event_type(event, self);
 	default:
 		self->unknown_events++;
 		return -1;
tools/perf/util/session.h +2 -1
index 7365a24..ea9eb4a 100644
@@ -44,7 +44,8 @@ struct perf_event_ops {
 		 read,
 		 throttle,
 		 unthrottle,
-		 attr;
+		 attr,
+		 event_type;
 };
 
 /*
Signature · 1 line
1.6.4.GIT
#10 lore raw

RFC PATCH 5/7 perf: convert perf tracing data into a tracing_data event

In reply to Tom Zanussi

Bypasses the tracing_data perf header code and replaces it with a synthesized event and processing function that accomplishes the same thing, used when reading/writing perf data to/from a pipe.

The tracing data is pretty large, and this patch doesn't attempt to break it down into component events. The tracing_data event itself doesn't actually contain the tracing data, rather it arranges for the event processing code to skip over it after it's read, using the 'action' added in a previous patch; in this case the action is 'skip'.

---
 tools/perf/builtin-record.c        |   16 ++++++
 tools/perf/util/event.h            |    7 +++
 tools/perf/util/header.c           |   53 +++++++++++++++++++++
 tools/perf/util/header.h           |    6 ++
 tools/perf/util/session.c          |   14 ++++++
 tools/perf/util/session.h          |    3 +-
 tools/perf/util/trace-event-info.c |   24 ++++++++++
 tools/perf/util/trace-event-read.c |   89 ++++++++++++++++++------------------
 tools/perf/util/trace-event.h      |    4 +-
 9 files changed, 170 insertions(+), 46 deletions(-)
tools/perf/builtin-record.c +16 -0
index 0e3d4cd..95bccdc 100644
@@ -101,6 +101,11 @@ static void mmap_write_tail(struct mmap_data *md, unsigned long tail)
 	pc->data_tail = tail;
 }
 
+static void advance_output(size_t size)
+{
+	bytes_written += size;
+}
+
 static void write_output(void *buf, size_t size)
 {
 	while (size) {
@@ -581,6 +586,17 @@ static int __cmd_record(int argc, const char **argv)
 			pr_err("Couldn't synthesize event_types.\n");
 			return err;
 		}
+
+		err = event__synthesize_tracing_data(output, attrs,
+						     nr_counters,
+						     process_synthesized_event,
+						     session);
+		if (err <= 0) {
+			pr_err("Couldn't record tracing data.\n");
+			return err;
+		}
+
+		advance_output(err);
 	}
 
 	err = event__synthesize_kernel_mmap(process_synthesized_event,
tools/perf/util/event.h +7 -0
index dc6d27b..c111358 100644
@@ -86,6 +86,7 @@ struct build_id_event {
 enum perf_header_event_type { /* above any possible kernel type */
 	PERF_RECORD_HEADER_ATTR			= 64,
 	PERF_RECORD_HEADER_EVENT_TYPE		= 65,
+	PERF_RECORD_HEADER_TRACING_DATA		= 66,
 	PERF_RECORD_HEADER_MAX
 };
 
@@ -107,6 +108,11 @@ struct event_type_event {
 	struct perf_trace_event_type event_type;
 };
 
+struct tracing_data_event {
+	struct perf_event_header header;
+	u32 size;
+};
+
 typedef union event_union {
 	struct perf_event_header	header;
 	struct ip_event			ip;
@@ -118,6 +124,7 @@ typedef union event_union {
 	struct sample_event		sample;
 	struct attr_event		attr;
 	struct event_type_event		event_type;
+	struct tracing_data_event	tracing_data;
 } event_t;
 
 struct events_stats {
tools/perf/util/header.c +53 -0

62 lines not on this page · raw message

tools/perf/util/header.h +6 -0
index 9d18c9b..a83f375 100644
@@ -111,5 +111,11 @@ int event__synthesize_event_types(event__handler_t process,
 int event__process_event_type(event_t *self,
 			      struct perf_session *session);
 
+int event__synthesize_tracing_data(int fd, struct perf_event_attr *pattrs,
+				   int nb_events,
+				   event__handler_t process,
+				   struct perf_session *session);
+int event__process_tracing_data(event_t *self,
+				struct perf_session *session);
 
 #endif /* __PERF_HEADER_H */
tools/perf/util/session.c +14 -0
index d704bdf..74e9646 100644
@@ -213,6 +213,8 @@ static void perf_event_ops__fill_defaults(struct perf_event_ops *handler)
 		handler->attr = process_event_stub;
 	if (handler->event_type == NULL)
 		handler->event_type = process_event_stub;
+	if (handler->tracing_data == NULL)
+		handler->tracing_data = process_event_stub;
 }
 
 static int process_event_stop(event_t *event __used,
@@ -235,6 +237,7 @@ void perf_event_ops__fill_stop(struct perf_event_ops *handler)
 	handler->unthrottle = process_event_stop;
 	handler->attr = process_event_stop;
 	handler->event_type = process_event_stop;
+	handler->tracing_data = process_event_stop;
 }
 
 static const char *event__name[] = {
@@ -250,6 +253,7 @@ static const char *event__name[] = {
 	[PERF_RECORD_SAMPLE]	 = "SAMPLE",
 	[PERF_RECORD_HEADER_ATTR]	 = "ATTR",
 	[PERF_RECORD_HEADER_EVENT_TYPE]	 = "EVENT_TYPE",
+	[PERF_RECORD_HEADER_TRACING_DATA]	 = "TRACING_DATA",
 };
 
 unsigned long event__total[PERF_RECORD_HEADER_MAX];
@@ -342,6 +346,11 @@ static void event__event_type_swap(event_t *self)
 		bswap_64(self->event_type.event_type.event_id);
 }
 
+static void event__tracing_data_swap(event_t *self)
+{
+	self->tracing_data.size = bswap_32(self->tracing_data.size);
+}
+
 typedef void (*event__swap_op)(event_t *self);
 
 static event__swap_op event__swap_ops[] = {
@@ -354,6 +363,7 @@ static event__swap_op event__swap_ops[] = {
 	[PERF_RECORD_SAMPLE] = event__all64_swap,
 	[PERF_RECORD_HEADER_ATTR]   = event__attr_swap,
 	[PERF_RECORD_HEADER_EVENT_TYPE]   = event__event_type_swap,
+	[PERF_RECORD_HEADER_TRACING_DATA]   = event__tracing_data_swap,
 	[PERF_RECORD_HEADER_MAX]    = NULL,
 };
 
@@ -398,6 +408,10 @@ static int perf_session__process_event(struct perf_session *self,
 		return ops->attr(event, self);
 	case PERF_RECORD_HEADER_EVENT_TYPE:
 		return ops->event_type(event, self);
+	case PERF_RECORD_HEADER_TRACING_DATA:
+		/* setup for reading amidst mmap */
+		lseek(self->fd, offset + head, SEEK_SET);
+		return ops->tracing_data(event, self);
 	default:
 		self->unknown_events++;
 		return -1;
tools/perf/util/session.h +2 -1
index ea9eb4a..a648af0 100644
@@ -45,7 +45,8 @@ struct perf_event_ops {
 		 throttle,
 		 unthrottle,
 		 attr,
-		 event_type;
+		 event_type,
+		 tracing_data;
 };
 
 /*
tools/perf/util/trace-event-info.c +24 -0
index 5ea8973..30cd9b5 100644
@@ -154,10 +154,17 @@ static void put_tracing_file(char *file)
 	free(file);
 }
 
+static ssize_t calc_data_size;
+
 static ssize_t write_or_die(const void *buf, size_t len)
 {
 	int ret;
 
+	if (calc_data_size) {
+		calc_data_size += len;
+		return len;
+	}
+
 	ret = write(output_fd, buf, len);
 	if (ret < 0)
 		die("writing to '%s'", output_file);
@@ -526,3 +533,20 @@ int read_tracing_data(int fd, struct perf_event_attr *pattrs, int nb_events)
 
 	return 0;
 }
+
+ssize_t read_tracing_data_size(int fd, struct perf_event_attr *pattrs,
+			       int nb_events)
+{
+	ssize_t size;
+	int err = 0;
+
+	calc_data_size = 1;
+	err = read_tracing_data(fd, pattrs, nb_events);
+	size = calc_data_size - 1;
+	calc_data_size = 0;
+
+	if (err < 0)
+		return err;
+
+	return size;
+}
tools/perf/util/trace-event-read.c +45 -44

142 lines not on this page · raw message

tools/perf/util/trace-event.h +3 -1
index c3269b9..0cb5866 100644
@@ -162,7 +162,7 @@ struct record *trace_read_data(int cpu);
 
 void parse_set_info(int nr_cpus, int long_sz);
 
-void trace_report(int fd);
+ssize_t trace_report(int fd);
 
 void *malloc_or_die(unsigned int size);
 
@@ -258,6 +258,8 @@ void *raw_field_ptr(struct event *event, const char *name, void *data);
 unsigned long long eval_flag(const char *flag);
 
 int read_tracing_data(int fd, struct perf_event_attr *pattrs, int nb_events);
+ssize_t read_tracing_data_size(int fd, struct perf_event_attr *pattrs,
+			       int nb_events);
 
 /* taken from kernel/trace/trace.h */
 enum trace_flag_type {
Signature · 1 line
1.6.4.GIT
#11 lore raw

RFC PATCH 6/7 perf: convert perf header build_ids into build_id events

In reply to Tom Zanussi

Bypasses the build_id perf header code and replaces it with a synthesized event and processing function that accomplishes the same thing, used when reading/writing perf data to/from a pipe.

---
 tools/perf/builtin-record.c |   15 ++++++-
 tools/perf/util/event.h     |    2 +
 tools/perf/util/header.c    |   90 +++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/header.h    |    7 +++
 tools/perf/util/session.c   |    7 +++
 tools/perf/util/session.h   |    3 +-
 6 files changed, 120 insertions(+), 4 deletions(-)
tools/perf/builtin-record.c +12 -3
index 95bccdc..0ab9ffa 100644
@@ -405,10 +405,19 @@ static int process_buildids(void)
 
 static void atexit_header(void)
 {
-	session->header.data_size += bytes_written;
+	if (!pipe_output) {
+		session->header.data_size += bytes_written;
 
-	process_buildids();
-	perf_header__write(&session->header, output, true);
+		process_buildids();
+		perf_header__write(&session->header, output, true);
+	} else {
+		int err;
+
+		err = event__synthesize_build_ids(process_synthesized_event,
+						  session);
+		if (err < 0)
+			pr_err("Couldn't synthesize build ids.\n");
+	}
 }
 
 static int __cmd_record(int argc, const char **argv)
tools/perf/util/event.h +2 -0
index c111358..0705e75 100644
@@ -87,6 +87,7 @@ enum perf_header_event_type { /* above any possible kernel type */
 	PERF_RECORD_HEADER_ATTR			= 64,
 	PERF_RECORD_HEADER_EVENT_TYPE		= 65,
 	PERF_RECORD_HEADER_TRACING_DATA		= 66,
+	PERF_RECORD_HEADER_BUILD_ID		= 67,
 	PERF_RECORD_HEADER_MAX
 };
 
@@ -125,6 +126,7 @@ typedef union event_union {
 	struct attr_event		attr;
 	struct event_type_event		event_type;
 	struct tracing_data_event	tracing_data;
+	struct build_id_event		build_id;
 } event_t;
 
 struct events_stats {
tools/perf/util/header.c +90 -0

93 lines not on this page · raw message

tools/perf/util/header.h +7 -0
index a83f375..cfb807f 100644
@@ -118,4 +118,11 @@ int event__synthesize_tracing_data(int fd, struct perf_event_attr *pattrs,
 int event__process_tracing_data(event_t *self,
 				struct perf_session *session);
 
+int event__synthesize_build_id(struct dso *pos, u16 misc,
+			       event__handler_t process,
+			       struct perf_session *session);
+int event__synthesize_build_ids(event__handler_t process,
+				struct perf_session *session);
+int event__process_build_id(event_t *self, struct perf_session *session);
+
 #endif /* __PERF_HEADER_H */
tools/perf/util/session.c +7 -0
index 74e9646..679274d 100644
@@ -215,6 +215,8 @@ static void perf_event_ops__fill_defaults(struct perf_event_ops *handler)
 		handler->event_type = process_event_stub;
 	if (handler->tracing_data == NULL)
 		handler->tracing_data = process_event_stub;
+	if (handler->build_id == NULL)
+		handler->build_id = process_event_stub;
 }
 
 static int process_event_stop(event_t *event __used,
@@ -238,6 +240,7 @@ void perf_event_ops__fill_stop(struct perf_event_ops *handler)
 	handler->attr = process_event_stop;
 	handler->event_type = process_event_stop;
 	handler->tracing_data = process_event_stop;
+	handler->build_id = process_event_stop;
 }
 
 static const char *event__name[] = {
@@ -254,6 +257,7 @@ static const char *event__name[] = {
 	[PERF_RECORD_HEADER_ATTR]	 = "ATTR",
 	[PERF_RECORD_HEADER_EVENT_TYPE]	 = "EVENT_TYPE",
 	[PERF_RECORD_HEADER_TRACING_DATA]	 = "TRACING_DATA",
+	[PERF_RECORD_HEADER_BUILD_ID]	 = "BUILD_ID",
 };
 
 unsigned long event__total[PERF_RECORD_HEADER_MAX];
@@ -364,6 +368,7 @@ static event__swap_op event__swap_ops[] = {
 	[PERF_RECORD_HEADER_ATTR]   = event__attr_swap,
 	[PERF_RECORD_HEADER_EVENT_TYPE]   = event__event_type_swap,
 	[PERF_RECORD_HEADER_TRACING_DATA]   = event__tracing_data_swap,
+	[PERF_RECORD_HEADER_BUILD_ID]   = NULL,
 	[PERF_RECORD_HEADER_MAX]    = NULL,
 };
 
@@ -412,6 +417,8 @@ static int perf_session__process_event(struct perf_session *self,
 		/* setup for reading amidst mmap */
 		lseek(self->fd, offset + head, SEEK_SET);
 		return ops->tracing_data(event, self);
+	case PERF_RECORD_HEADER_BUILD_ID:
+		return ops->build_id(event, self);
 	default:
 		self->unknown_events++;
 		return -1;
tools/perf/util/session.h +2 -1
index a648af0..87821a9 100644
@@ -46,7 +46,8 @@ struct perf_event_ops {
 		 unthrottle,
 		 attr,
 		 event_type,
-		 tracing_data;
+		 tracing_data,
+		 build_id;
 };
 
 /*
Signature · 1 line
1.6.4.GIT
#12 lore raw

RFC PATCH 7/7 perf trace/scripting: rwtop and sctop scripts

In reply to Tom Zanussi

A couple of scripts, one in Python and the other in Perl, that demonstrate 'live mode' tracing. For each, the output of the perf event stream is fed continuously to the script, which continuously aggregates the data and reports the current results every 5 seconds. After the current results are displayed, the aggregations are cleared and the cycle begins anew.

To run the scripts, simply pipe the output of the 'perf trace record' step as input to the corresponding 'perf trace report' step:

$ perf trace record sctop | perf trace report sctop

A future patch should probably automatically do this i.e. the user should be able to just run 'perf trace sctop' and get the same thing, and the output should do better than just continuously scroll, but these are just demo scripts for now...

---
 tools/perf/scripts/perl/bin/rwtop-record   |    2 +
 tools/perf/scripts/perl/bin/rwtop-report   |    6 +
 tools/perf/scripts/perl/rwtop.pl           |  165 ++++++++++++++++++++++++++++
 tools/perf/scripts/python/bin/sctop-record |    2 +
 tools/perf/scripts/python/bin/sctop-report |    4 +
 tools/perf/scripts/python/sctop.py         |   64 +++++++++++
 6 files changed, 243 insertions(+), 0 deletions(-)
 create mode 100644 tools/perf/scripts/perl/bin/rwtop-record
 create mode 100644 tools/perf/scripts/perl/bin/rwtop-report
 create mode 100644 tools/perf/scripts/perl/rwtop.pl
 create mode 100644 tools/perf/scripts/python/bin/sctop-record
 create mode 100644 tools/perf/scripts/python/bin/sctop-report
 create mode 100644 tools/perf/scripts/python/sctop.py
tools/perf/scripts/perl/bin/rwtop-record new file +2 -0
new file mode 100644
index 0000000..0993193
@@ -0,0 +1,2 @@
+#!/bin/bash
+perf record -o - -c 1 -f -a -M -R -e syscalls:sys_enter_read -e syscalls:sys_exit_read -e syscalls:sys_enter_write -e syscalls:sys_exit_write
tools/perf/scripts/perl/bin/rwtop-report new file +6 -0
new file mode 100644
index 0000000..3d06c98
@@ -0,0 +1,6 @@
+#!/bin/bash
+# description: system-wide r/w top
+perf trace -i - -s ~/libexec/perf-core/scripts/perl/rwtop.pl
+
+
+
tools/perf/scripts/perl/rwtop.pl new file +165 -0

165 lines not on this page · raw message

tools/perf/scripts/python/bin/sctop-record new file +2 -0
new file mode 100644
index 0000000..1a46fb6
@@ -0,0 +1,2 @@
+#!/bin/bash
+perf record -o - -c 1 -f -a -M -R -e raw_syscalls:sys_enter
tools/perf/scripts/python/bin/sctop-report new file +4 -0
new file mode 100644
index 0000000..4ca46c0
@@ -0,0 +1,4 @@
+#!/bin/bash
+# description: syscall top
+# args: [comm]
+perf trace -i - -s ~/libexec/perf-core/scripts/python/sctop.py $1
tools/perf/scripts/python/sctop.py new file +64 -0

64 lines not on this page · raw message

Signature · 1 line
1.6.4.GIT
#13 lore raw
In reply to Tom Zanussi
Quoted 7 lines · Tom Zanussi

Currently, a perf session entails two steps: first 'perf record' or 'perf trace record' records the perf data to disk, then 'perf report' or 'perf trace report' reads the saved data from disk and reports the results.

This experimental patchset makes some changes to perf that instead allow the perf data to be piped directly from the record step to the report step, without ever touching the disk.

Very nice!

Quoted 4 lines

Obviously, it would be better to have a real top-like display for these rather than a continuously scrolling mode like this, and of course it will be much more useful once we get the syscall name injection events going (the column on the left shows syscall numbers only).

It's still useful for ad-hoc tracing!

Side-note, it might make sense to expose the 'clear' escape sequence somehow in an easy fashion:



to make it non-scrolling ;-) via a pre-provided method.

Quoted 3 lines

There are some rough edges and inefficiencies, and I guess 'event injection' may be a better way to do this in the end, but it seems to work pretty well already, and I think it shows how useful such a capability can be.

I'm all for it. Frederic, Steve, what do you think?

Ingo

#14 lore raw

Em Thu, Mar 04, 2010 at 12:18:56PM +0100, Ingo Molnar escreveu:

Quoted 24 lines · Tom Zanussi
Quoted 8 lines

Currently, a perf session entails two steps: first 'perf record' or 'perf trace record' records the perf data to disk, then 'perf report' or 'perf trace report' reads the saved data from disk and reports the results.

This experimental patchset makes some changes to perf that instead allow the perf data to be piped directly from the record step to the report step, without ever touching the disk.

Very nice!

Quoted 4 lines

Obviously, it would be better to have a real top-like display for these rather than a continuously scrolling mode like this, and of course it will be much more useful once we get the syscall name injection events going (the column on the left shows syscall numbers only).

It's still useful for ad-hoc tracing!

Side-note, it might make sense to expose the 'clear' escape sequence somehow in an easy fashion:



Quoted 7 lines

to make it non-scrolling ;-) via a pre-provided method.

Quoted 3 lines

There are some rough edges and inefficiencies, and I guess 'event injection' may be a better way to do this in the end, but it seems to work pretty well already, and I think it shows how useful such a capability can be.

I'm all for it. Frederic, Steve, what do you think?

ITs really nice indeed, I'm reviewing the patches and will test them after weekend, it fits nicely with the idea of unifying top and report, i.e. the refresh thing he mentions could do the decay as well if a parameter is set.

Thnaks Frederic for pointing this patchset to me.

Great work!

- Arnaldo

#15 lore raw
Quoted 33 lines · Arnaldo Carvalho de Melo

Em Thu, Mar 04, 2010 at 12:18:56PM +0100, Ingo Molnar escreveu:

Quoted 24 lines · Tom Zanussi
Quoted 8 lines

Currently, a perf session entails two steps: first 'perf record' or 'perf trace record' records the perf data to disk, then 'perf report' or 'perf trace report' reads the saved data from disk and reports the results.

This experimental patchset makes some changes to perf that instead allow the perf data to be piped directly from the record step to the report step, without ever touching the disk.

Very nice!

Quoted 4 lines

Obviously, it would be better to have a real top-like display for these rather than a continuously scrolling mode like this, and of course it will be much more useful once we get the syscall name injection events going (the column on the left shows syscall numbers only).

It's still useful for ad-hoc tracing!

Side-note, it might make sense to expose the 'clear' escape sequence somehow in an easy fashion:



Quoted 7 lines

to make it non-scrolling ;-) via a pre-provided method.

Quoted 3 lines

There are some rough edges and inefficiencies, and I guess 'event injection' may be a better way to do this in the end, but it seems to work pretty well already, and I think it shows how useful such a capability can be.

I'm all for it. Frederic, Steve, what do you think?

Quoted 8 lines

ITs really nice indeed, I'm reviewing the patches and will test them after weekend, it fits nicely with the idea of unifying top and report, i.e. the refresh thing he mentions could do the decay as well if a parameter is set.

Thnaks Frederic for pointing this patchset to me.

Great work!

Thanks! Just FYI, I have a new version of this patchset that I'm hoping to post in the next couple of days. It adds the refresh sequence and a specifiable delay, among other things (like bugfixes)...

Tom

Quoted 2 lines

- Arnaldo

#16 lore raw
In reply to Tom Zanussi
Quoted 176 lines · Tom Zanussi

Currently, a perf session entails two steps: first 'perf record' or 'perf trace record' records the perf data to disk, then 'perf report' or 'perf trace report' reads the saved data from disk and reports the results.

This experimental patchset makes some changes to perf that instead allow the perf data to be piped directly from the record step to the report step, without ever touching the disk.

This is especially useful for 'perf trace' - adding this capability means that the trace scripts are no longer relegated to simple post-processing, but can be run in a continuous 'live mode', forever processing the event stream and e.g. periodically dumping current results, essentially becoming special-purpose 'top' applications, or continuously scanning the event stream for arbitrarily complex conditions to flag, etc...

Being able to feed the event stream over a pipe also makes it possible to do things like trace over the network using e.g. netcat.

It turns out that perf is pretty close to being able to do this already, with the exception of the header data; most of the work of this patchset deals with changing that. It does so in a mainly additive way: it doesn't make any changes to the existing disk format or normal disk-mode processing, just adds special treatment for the cases when 'perf [trace] record' records to stdout or 'perf [trace] report reads from stdin.

Here are some quick examples. Basically using '-' as the filename to the -o and -i options send/read the data from stdout/stdin:

Here's a simple 'perf record' | 'perf report' run:

root@tropicana:~# perf record -o - -c 1 -f -a -M -R -e raw_syscalls:sys_enter | perf report -i -
^C# Samples: 120234
#
# Overhead          Command              Shared Object  Symbol
# ........  ...............  .........................  ......
#
    98.65%             perf  libpthread-2.8.90.so       [.] __read
     0.46%             perf  libpthread-2.8.90.so       [.] __write_nocancel
     0.32%             perf  libpthread-2.8.90.so       [.] __open_nocancel
     0.29%             perf  libpthread-2.8.90.so       [.] __libc_close
     0.07%             perf  libc-2.8.90.so             [.] 0x000000000a4b15
     0.03%             perf  libc-2.8.90.so             [.] __fxstat64
     0.03%             perf  libc-2.8.90.so             [.] __open
     0.03%             perf  libc-2.8.90.so             [.] close
     0.02%             Xorg  libc-2.8.90.so             [.] __select
     0.01%             Xorg  libpthread-2.8.90.so       [.] __read
     0.01%             perf  libpthread-2.8.90.so       [.] __libc_lseek
     0.01%             Xorg  libc-2.8.90.so             [.] setitimer
     0.01%             perf  libc-2.8.90.so             [.] __xstat64
     0.01%          firefox  libpthread-2.8.90.so       [.] __read
     0.00%             perf  libc-2.8.90.so             [.] llseek
     0.00%             Xorg  libpthread-2.8.90.so       [.] __restore_rt
     0.00%             Xorg  libc-2.8.90.so             [.] sigprocmask
     0.00%             Xorg  libc-2.8.90.so             [.] __poll
     0.00%            wterm  libc-2.8.90.so             [.] __read
     0.00%  gnome-screensav  libpthread-2.8.90.so       [.] __read
     0.00%          firefox  libc-2.8.90.so             [.] __poll
     0.00%             Xorg  libc-2.8.90.so             [.] writev
     0.00%  gnome-settings-  libpthread-2.8.90.so       [.] __read
     0.00%            wterm  libc-2.8.90.so             [.] __select
     0.00%             perf  libc-2.8.90.so             [.] statfs
     0.00%           mysqld  libc-2.8.90.so             [.] __select
     0.00%          firefox  libpthread-2.8.90.so       [.] __pthread_cond_timedwait
     0.00%          firefox  libpthread-2.8.90.so       [.] __lll_unlock_wake
     0.00%          firefox  libpthread-2.8.90.so       [.] 0x0000000000decb
     0.00%          firefox  libc-2.8.90.so             [.] __select
     0.00%             perf  libc-2.8.90.so             [.] getcwd
     0.00%             perf  libc-2.8.90.so             [.] mmap64
     0.00%  gnome-settings-  libc-2.8.90.so             [.] __poll
     0.00%  gnome-screensav  libc-2.8.90.so             [.] __poll
     0.00%          firefox  libc-2.8.90.so             [.] writev
     0.00%          apache2  libpthread-2.8.90.so       [.] __waitpid
     0.00%          apache2  libc-2.8.90.so             [.] __select
#
# (For a higher level overview, try: perf report --sort comm,dso)
#

Included in this patchset are a couple of 'top' scripts, rwtop and sctop, that are essentially just modifications of existing scripts. Basically the original scripts were modified to add a 5 second timer. In the handler for the timer, the current output summary is printed, and the state is cleared and begun anew, ad infinitum.

Here are the new scripts as shown in the perf trace list:

root@tropicana:~# perf trace -l
List of available trace scripts:
  workqueue-stats                      workqueue stats (ins/exe/create/destroy)
  wakeup-latency                       system-wide min/max/avg wakeup latency
  rw-by-file <comm>                    r/w activity for a program, by file
  rwtop                                system-wide r/w top
  failed-syscalls [comm]               system-wide failed syscalls
  rw-by-pid                            system-wide r/w activity
  syscall-counts-by-pid [comm]         system-wide syscall counts, by pid
  failed-syscalls-by-pid [comm]        system-wide failed syscalls, by pid
  sctop [comm]                         syscall top
  syscall-counts [comm]                system-wide syscall counts

And here's a few iterations of the output of the 'sctop' Python script:

root@tropicana:~# perf trace record sctop | perf trace report sctop
perf trace started with Python script /root/libexec/perf-core/scripts/python/sctop.py

syscall events:

event count
---------------------------------------- -----------
2 179
3 158

0                                                 79
78                                                24
5                                                 13
1                                                  3
16                                                 2
298                                                1
137                                                1
79                                                 1
72                                                 1
8                                                  1

syscall events:

event count
---------------------------------------- -----------
0 659287
202 1995

1                                                812
2                                                246
3                                                224
78                                                56
12                                                51
5                                                 23
23                                                22
8                                                 18
7                                                 18
4                                                  9
38                                                 6
61                                                 5
254                                                4
20                                                 2
16                                                 2
137                                                1
15                                                 1
14                                                 1
9                                                  1

syscall events:

event count
---------------------------------------- -----------
0 647178
202 1209
1 324
12 52
7 29

14                                                 6
16                                                 4
23                                                 3
3                                                  3
72                                                 2
21                                                 1
11                                                 1
9                                                  1
5                                                  1
4                                                  1
2                                                  1

.
.
.

Quoted 6 lines

Obviously, it would be better to have a real top-like display for these rather than a continuously scrolling mode like this, and of course it will be much more useful once we get the syscall name injection events going (the column on the left shows syscall numbers only).

That's a very nice thing. I guess that we could make the scripting API to provide something to write such top like things.

Providing a simple dict must be enough, something organized like this:

topdict = {
	"colname" : [val1, val2, val3],
	"col2name" : [val4, val5, val6]
}

Or may be a callback that returns a dict.