perf tools: Introduce new 'ftrace' command (v2)
Namhyung Kim
maintainercover perf tools: Introduce new 'ftrace' command (v2)
Hi all,
This patchset implements a front-end tool for kernel's ftrace. It uses function_graph tracer by default and normal function tracer is also supported. (Of course you need to enable those tracers in your kernel first.)
NOTE: It's in very early stage, so may contain many rough edges.
v2 changes:
* factor out tracepoint_name_to_path() function
* rework [re]set_tracing_cpumask() function to work with many cpus
* fix a deadlock in 'record' path
* fix a bug regarding to the pager setup
* add a debug patch for finding segfaults
I pushed it out to 'perf/ftrace-v2' branch on my tree at:
git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
Following is the original description and example.
----
It consists of 4 subcommands: live, record, show and report.
'perf ftrace live' just triggers ftrace and relay kernel buffer contents to stdout. It does no processing in the tool side.
'perf ftrace record' starts ftrace and saves its result to per-cpu files and a perf.header file in the perf.data.dir directory. Recording was done by multiple threads (a thread per cpu) in order not to miss events overrun. The perf.header file is compatible to current perf.data file and contains useful information and sample data.
The sample data were synthesized for each recorded cpu to provide more information - I'm not sure it's really needed though.
Once you had run 'perf ftrace record', you could play with other subcommands.
'perf ftrace show' displays function traces like 'live' subcommand or trace-cmd does. It's not useful than them at this time but it could be improved soon.
'perf ftrace report' displays usual 'perf report' style output from the function trace data. You can see which function is called most frequently for example. Currently it uses 1 as a period value for each entry but we might use funcgraph_exit->calltime to get proper overhead later.
Example below:
# perf ftrace record sleep 0.1 # ls -l perf.data.dir total 5568 -rw-r--r--. 1 root root 3514375 Apr 23 16:43 perf.header -rw-r--r--. 1 root root 90112 Apr 23 16:43 trace-cpu0.buf -rw-r--r--. 1 root root 0 Apr 23 16:43 trace-cpu1.buf -rw-r--r--. 1 root root 2093056 Apr 23 16:43 trace-cpu2.buf -rw-r--r--. 1 root root 0 Apr 23 16:43 trace-cpu3.buf
# perf ftrace show
overriding event (11) ftrace:funcgraph_entry with new print handler
overriding event (10) ftrace:funcgraph_exit with new print handler
sleep-11573 [000] 717758.728678862: funcgraph_entry: finish_task_switch() {
sleep-11573 [000] 717758.728682155: funcgraph_exit: }
sleep-11573 [000] 717758.728683099: funcgraph_entry: finish_wait() {
sleep-11573 [000] 717758.728683617: funcgraph_exit: }
sleep-11573 [000] 717758.728683993: funcgraph_entry: pipe_lock() {
sleep-11573 [000] 717758.728684425: funcgraph_entry: pipe_lock_nested.isra.
sleep-11573 [000] 717758.728684863: funcgraph_entry: mutex_lock() {
sleep-11573 [000] 717758.728685239: funcgraph_entry: _cond_resched() {
sleep-11573 [000] 717758.728685807: funcgraph_exit: }
sleep-11573 [000] 717758.728686296: funcgraph_exit: }
sleep-11573 [000] 717758.728686757: funcgraph_exit: }
sleep-11573 [000] 717758.728687529: funcgraph_exit: }
sleep-11573 [000] 717758.728688608: funcgraph_entry: mutex_unlock() {
sleep-11573 [000] 717758.728689343: funcgraph_exit: }
sleep-11573 [000] 717758.728690685: funcgraph_entry: __audit_syscall_exit() {
sleep-11573 [000] 717758.728691499: funcgraph_entry: path_put() {
sleep-11573 [000] 717758.728692147: funcgraph_entry: dput() {
sleep-11573 [000] 717758.728692845: funcgraph_exit: }
sleep-11573 [000] 717758.728693380: funcgraph_entry: mntput() {
sleep-11573 [000] 717758.728694021: funcgraph_exit: }
sleep-11573 [000] 717758.728694520: funcgraph_exit: }
sleep-11573 [000] 717758.728695005: funcgraph_entry: unroll_tree_refs() {
sleep-11573 [000] 717758.728695659: funcgraph_exit: }
sleep-11573 [000] 717758.728696301: funcgraph_entry: kfree() {
sleep-11573 [000] 717758.728697085: funcgraph_exit: }
sleep-11573 [000] 717758.728697916: funcgraph_exit: } # perf ftrace report
# ========
# captured on: Tue Apr 23 16:43:52 2013
# hostname : sejong.aot.lge.com
# os release : 3.6.11-5.fc17.x86_64
# perf version : 3.9.rc8.gceb5f8
# arch : x86_64
# nrcpus online : 12
# nrcpus avail : 12
# cpudesc : Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
# cpuid : GenuineIntel,6,45,7
# total memory : 32901952 kB
# cmdline : /home/namhyung/project/linux/tools/perf/perf ftrace record sleep 0.1
# event : name = ftrace:funcgraph_entry, type = 2, config = 0xb, config1 = 0x0, config2 = 0x0
# event : name = ftrace:funcgraph_exit, type = 2, config = 0xa, config1 = 0x0, config2 = 0x0,
# HEADER_CPU_TOPOLOGY info available, use -I to display
# HEADER_NUMA_TOPOLOGY info available, use -I to display
# ========
#
# Samples: 22K of event 'ftrace:funcgraph_entry'
# Event count (approx.): 22433
#
# Overhead Command Shared Object Symbol
# ........ ....... ................. ..............................
#
7.90% sleep [kernel.kallsyms] [k] _cond_resched
4.57% sleep [kernel.kallsyms] [k] ebitmap_get_bit
3.50% sleep [kernel.kallsyms] [k] __phys_addr
3.12% sleep [kernel.kallsyms] [k] _raw_spin_lock
3.09% sleep [kernel.kallsyms] [k] kmem_cache_free
2.93% sleep [kernel.kallsyms] [k] __slab_free
2.86% sleep [kernel.kallsyms] [k] __tlb_remove_page
2.35% sleep [kernel.kallsyms] [k] vm_normal_page
2.33% sleep [kernel.kallsyms] [k] page_remove_rmap
1.87% sleep [kernel.kallsyms] [k] mutex_unlock Note that this patchset is based on my previous work [1][2].
Any comments are welcome, thanks,
Namhyung
[1] https://lkml.org/lkml/2013/4/11/194
[2] https://lkml.org/lkml/2013/4/15/13
Namhyung Kim (15):
perf util: Move debugfs/tracing helper functions to util.c perf util: Use evsel->name to get tracepoint_paths perf util: Save pid-cmdline mapping into tracing header perf util: Add more debug message on failure path perf tools: Introduce new 'ftrace' tool
perf ftrace: Add support for --pid option
perf ftrace: Add support for -a and -C option perf ftrace: Split "live" sub-command
perf ftrace: Add 'record' sub-command
perf ftrace: Add 'show' sub-command
perf ftrace: Add 'report' sub-command
perf ftrace: Use pager for displaying result perf ftrace: Cleanup using ftrace_setup/teardown() perf tools: Add document for perf-ftrace command perf ftrace: Add a signal handler for SIGSEGV
tools/perf/Documentation/perf-ftrace.txt | 114 ++ tools/perf/Makefile | 1 + tools/perf/builtin-ftrace.c | 1714 ++++++++++++++++++++++++++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/perf.c | 1 + tools/perf/util/cpumap.c | 45 + tools/perf/util/cpumap.h | 1 + tools/perf/util/header.c | 4 +- tools/perf/util/parse-events.c | 23 + tools/perf/util/parse-events.h | 1 + tools/perf/util/trace-event-info.c | 103 +- tools/perf/util/trace-event-parse.c | 17 + tools/perf/util/trace-event-read.c | 77 +- tools/perf/util/trace-event.h | 1 + tools/perf/util/util.c | 59 + tools/perf/util/util.h | 3 + 17 files changed, 2092 insertions(+), 74 deletions(-) create mode 100644 tools/perf/Documentation/perf-ftrace.txt create mode 100644 tools/perf/builtin-ftrace.c
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 1/15 perf util: Move debugfs/tracing helper functions to util.c
From: Namhyung Kim <[email protected]>
Since they're generic helpers move them to util.c so that they can be used by others.
--- tools/perf/util/trace-event-info.c | 59 -------------------------------------- tools/perf/util/util.c | 59 ++++++++++++++++++++++++++++++++++++++ tools/perf/util/util.h | 3 ++ 3 files changed, 62 insertions(+), 59 deletions(-)
tools/perf/util/trace-event-info.c +0 -59
@@ -46,65 +46,6 @@static int output_fd; -static const char *find_debugfs (void ) -{ -const char *path =perf_debugfs_mount (NULL ); - -if (!path) -pr_debug ("Your kernel does not support the debugfs filesystem" ); - -return path; -} - -/* -* Finds the path to the debugfs/tracing -* Allocates the string and stores it. -*/ -static const char *find_tracing_dir (void ) -{ -static char *tracing; -static int tracing_found; -const char *debugfs; - -if (tracing_found) -return tracing; - - debugfs =find_debugfs (); -if (!debugfs) -return NULL ; - - tracing =malloc (strlen (debugfs) +9 ); -if (!tracing) -return NULL ; - -sprintf (tracing,"%s/tracing" , debugfs); - - tracing_found =1 ; -return tracing; -} - -static char *get_tracing_file (const char *name) -{ -const char *tracing; -char *file; - - tracing =find_tracing_dir (); -if (!tracing) -return NULL ; - - file =malloc (strlen (tracing) +strlen (name) +2 ); -if (!file) -return NULL ; - -sprintf (file,"%s/%s" , tracing, name); -return file; -} - -static void put_tracing_file (char *file) -{ -free (file); -} -int bigendian (void ) {unsigned char str[] = {0x1 ,0x2 ,0x3 ,0x4 ,0x0 ,0x0 ,0x0 ,0x0 };
tools/perf/util/util.c +59 -0
62 lines not on this page · raw message
tools/perf/util/util.h +3 -0
@@ -79,6 +79,9 @@ extern char buildid_dir[];extern char tracing_events_path[];extern void perf_debugfs_set_path (const char *mountpoint);const char *perf_debugfs_mount (const char *mountpoint); +const char *find_tracing_dir (void ); +char *get_tracing_file (const char *name); +void put_tracing_file (char *file);/* On most systems <limits.h> would have given us this, but * not on some systems (e.g. GNU/Hurd).
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 2/15 perf util: Use evsel->name to get tracepoint_paths
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
Most tracepoint events already have their system and event name in ->name field so that searching whole event tracing directory for each evsel to match given id is suboptimal.
Factor out this routine into tracepoint_name_to_path().
--- tools/perf/util/parse-events.c | 23 +++++++++++++++++++++++ tools/perf/util/parse-events.h | 1 + tools/perf/util/trace-event-info.c | 11 +++++++++++ 3 files changed, 35 insertions(+)
tools/perf/util/parse-events.c +23 -0
@@ -217,6 +217,29 @@ struct tracepoint_path *tracepoint_id_to_path(u64 config)return NULL ; } +struct tracepoint_path *tracepoint_name_to_path (const char *name) +{ +struct tracepoint_path *path =zalloc (sizeof (*path)); +char *str =strchr (name,':' ); + +if (path ==NULL || str ==NULL ) { +free (path); +return NULL ; + } + + path->system =strndup (name, str - name); + path->name =strdup (str+1 ); + +if (path->system ==NULL || path->name ==NULL ) { +free (path->system); +free (path->name); +free (path); + path =NULL ; + } + +return path; +} +const char *event_type (int type) {switch (type) {
tools/perf/util/parse-events.h +1 -0
@@ -23,6 +23,7 @@ struct tracepoint_path { };extern struct tracepoint_path *tracepoint_id_to_path (u64 config); +extern struct tracepoint_path *tracepoint_name_to_path (const char *name);extern bool have_tracepoints (struct list_head *evlist);const char *event_type (int type);
tools/perf/util/trace-event-info.c +11 -0
@@ -414,12 +414,23 @@ get_tracepoints_path(struct list_head *pattrs)if (pos->attr.type !=PERF_TYPE_TRACEPOINT )continue ; ++nr_tracepoints; + +if (pos->name) { + ppath->next =tracepoint_name_to_path (pos->name); +if (!ppath->next) + goto error; + + goto next; + } + ppath->next =tracepoint_id_to_path (pos->attr.config);if (!ppath->next) { +error:pr_debug ("No memory to alloc tracepoints list\n" );put_tracepoints_path (&path);return NULL ; } +next: ppath = ppath->next; }
Signature · 1 line
1.7.11.7
Re: [PATCH 02/15] perf util: Use evsel->name to get tracepoint_paths
Quoted 1 line · Namhyung Kim
From: Namhyung Kim <[email protected]>
SNIP
Quoted 12 lines
};
extern struct
tracepoint_path*tracepoint_id_to_path(u64 config);tools/perf/util/parse-events.h +1
-0+extern struct tracepoint_path *tracepoint_name_to_path (const char *name);extern bool have_tracepoints (struct list_head *evlist);const char *event_type (int type);tools/perf/util/trace-event-info.c +0
-0
tools/perf/util/trace-event-info.c get_tracepoints_path(struct list_head *pattrs) · line 419
Quoted 6 lines
tools/perf/util/trace-event-info.c +3
-0@@ -414,12 +414,23 @@ get_tracepoints_path(struct list_head *pattrs)if (pos->attr.type !=PERF_TYPE_TRACEPOINT )continue ; ++nr_tracepoints; + +if (pos->name) { + ppath->next =tracepoint_name_to_path (pos->name);
yep, looks like big time saver ;)
tools/perf/util/trace-event-info.c get_tracepoints_path(struct list_head *pattrs) · line 421
Quoted 2 lines
tools/perf/util/trace-event-info.c +2
-0+if (!ppath->next) + goto error;
hum, the tracepoint_name_to_path fails also because of the malformed tracepoint name.. then the error message is misleading.. but not quite sure this could happen
jirka
Quoted 18 lines
tools/perf/util/trace-event-info.c +6
-0+ + goto next; + } + ppath->next =tracepoint_id_to_path (pos->attr.config);if (!ppath->next) { +error:pr_debug ("No memory to alloc tracepoints list\n" );put_tracepoints_path (&path);return NULL ; } +next: ppath = ppath->next; }Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerRe: [PATCH 02/15] perf util: Use evsel->name to get tracepoint_paths
Hi Jiri,
Quoted 15 lines · Jiri Olsa
tools/perf/util/trace-event-info.c get_tracepoints_path(struct list_head *pattrs) · line 419
Quoted 11 lines · Namhyung Kim
tools/perf/util/trace-event-info.c +3
-0@@ -414,12 +414,23 @@ get_tracepoints_path(struct list_head *pattrs)if (pos->attr.type !=PERF_TYPE_TRACEPOINT )continue ; ++nr_tracepoints; + +if (pos->name) { + ppath->next =tracepoint_name_to_path (pos->name);yep, looks like big time saver ;)
Quoted 6 lines
tools/perf/util/trace-event-info.c get_tracepoints_path(struct list_head *pattrs) · line 421
Quoted 2 lines
tools/perf/util/trace-event-info.c +2
-0+if (!ppath->next) + goto error;hum, the
tracepoint_name_to_pathfails also because of the malformed tracepoint name.. then the error message is misleading.. but not quite sure this could happen
AFAIK it's called on the record path so every event is generated by perf_evsel__newtp() and has a proper name. But there's a way to specify events using numeric form.. :(
So yes, I think it's better to consider such a exceptional case.
Thanks,
Namhyung
Namhyung Kim
maintainerPATCH 3/15 perf util: Save pid-cmdline mapping into tracing header
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
Current trace info data lacks the saved cmdline mapping which is needed for pevent to find out the comm of a task. Add this and bump up the version number so that perf can determine its presence when reading.
This is mostly corresponding to trace.dat file version 6, but still lacks 4 byte of number of cpus, and 10 bytes of type string - and I think we don't need those anyway.
--- tools/perf/util/trace-event-info.c | 33 ++++++++++++++++++++++++++++++++- tools/perf/util/trace-event-parse.c | 17 +++++++++++++++++ tools/perf/util/trace-event-read.c | 36 ++++++++++++++++++++++++++++++++++-- tools/perf/util/trace-event.h | 1 + 4 files changed, 84 insertions(+), 3 deletions(-)
tools/perf/util/trace-event-info.c +32 -1
@@ -41,7 +41,7 @@#include <lk/debugfs.h> #include "evsel.h" -#define VERSION "0.5" +#define VERSION "0.6"static int output_fd;
@@ -390,6 +390,34 @@ out:return err; } +static int record_saved_cmdline (void ) +{ +unsigned int size; +char *path; +struct stat st; +int ret, err =0 ; + + path =get_tracing_file ("saved_cmdlines" ); +if (!path) { +pr_debug ("can't get tracing/saved_cmdline" ); +return -ENOMEM ; + } + + ret =stat (path, &st); +if (ret <0 ) { +/* not found */ + size =0 ; +if (write (output_fd, &size,8 ) !=8 ) + err = -EIO ; + goto out; + } + err =record_file (path,8 ); + +out: +put_tracing_file (path); +return err; +} +static void put_tracepoints_path (struct tracepoint_path *tps) {
@@ -546,6 +574,9 @@ struct tracing_data *tracing_data_get(struct list_head *pattrs,if (err) goto out; err =record_ftrace_printk (); +if (err) + goto out; + err =record_saved_cmdline (); out:/*
tools/perf/util/trace-event-parse.c +17 -0
@@ -232,6 +232,23 @@ void parse_ftrace_printk(struct pevent *pevent, } } +void parse_saved_cmdline (struct pevent *pevent, +char *file,unsigned int size__maybe_unused ) +{ +char *comm; +char *line; +char *next =NULL ; +int pid; + + line =strtok_r (file,"\n" , &next); +while (line) { +sscanf (line,"%d %ms" , &pid, &comm); +pevent_register_comm (pevent, comm, pid); +free (comm); + line =strtok_r (NULL ,"\n" , &next); + } +} +int parse_ftrace_file (struct pevent *pevent,char *buf,unsigned long size) {return pevent_parse_event (pevent, buf, size,"ftrace" );
tools/perf/util/trace-event-read.c +34 -2
@@ -343,6 +343,31 @@ static int read_event_files(struct pevent *pevent)return 0 ; } +static int read_saved_cmdline (struct pevent *pevent) +{ +unsigned long long size; +char *buf; + +/* it can have 0 size */ + size =read8 (pevent); +if (!size) +return 0 ; + + buf =malloc (size +1 ); +if (buf ==NULL ) +return -1 ; + +if (do_read (buf, size) <0 ) { +free (buf); +return -1 ; + } + +parse_saved_cmdline (pevent, buf, size); + +free (buf); +return 0 ; +} +ssize_t trace_report (int fd,struct pevent **ppevent,bool __repipe) {char buf[BUFSIZ ];
@@ -383,10 +408,11 @@ ssize_t trace_report(int fd, struct pevent **ppevent, bool __repipe)return -1 ;if (show_version)printf ("version = %s\n" , version); -free (version); -if (do_read (buf,1 ) <0 ) +if (do_read (buf,1 ) <0 ) { +free (version);return -1 ; + } file_bigendian = buf[0 ]; host_bigendian =bigendian ();
@@ -422,6 +448,11 @@ ssize_t trace_report(int fd, struct pevent **ppevent, bool __repipe) err =read_ftrace_printk (pevent);if (err) goto out; +if (!strcmp (version,"0.6" )) { + err =read_saved_cmdline (pevent); +if (err) + goto out; + } size = trace_data_size; repipe = false;
@@ -438,5 +469,6 @@ ssize_t trace_report(int fd, struct pevent **ppevent, bool __repipe) out:if (pevent)pevent_free (pevent); +free (version);return size; }
tools/perf/util/trace-event.h +1 -0
@@ -30,6 +30,7 @@ void *raw_field_ptr(struct event_format *event, const char *name, void *data);void parse_proc_kallsyms (struct pevent *pevent,char *file,unsigned int size);void parse_ftrace_printk (struct pevent *pevent,char *file,unsigned int size); +void parse_saved_cmdline (struct pevent *pevent,char *file,unsigned int size);ssize_t trace_report (int fd,struct pevent **pevent,bool repipe);
Signature · 1 line
1.7.11.7
Re: [PATCH 03/15] perf util: Save pid-cmdline mapping into tracing header
Quoted 83 lines · Namhyung Kim
From: Namhyung Kim <[email protected]>
Current trace info data lacks the saved cmdline mapping which is needed for pevent to find out the comm of a task. Add this and bump up the version number so that perf can determine its presence when reading.
This is mostly corresponding to trace.dat file version 6, but still lacks 4 byte of number of cpus, and 10 bytes of type string - and I think we don't need those anyway.
Cc Steven Rostedt <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/trace-event-info.c | 33 ++++++++++++++++++++++++++++++++- tools/perf/util/trace-event-parse.c | 17 +++++++++++++++++ tools/perf/util/trace-event-read.c | 36 ++++++++++++++++++++++++++++++++++-- tools/perf/util/trace-event.h | 1 + 4 files changed, 84 insertions(+), 3 deletions(-)tools/perf/util/trace-event-info.c +32
-1@@ -41,7 +41,7 @@#include <lk/debugfs.h> #include "evsel.h" -#define VERSION "0.5" +#define VERSION "0.6"static int output_fd;@@ -390,6 +390,34 @@ out:return err; } +static int record_saved_cmdline (void ) +{ +unsigned int size; +char *path; +struct stat st; +int ret, err =0 ; + + path =get_tracing_file ("saved_cmdlines" ); +if (!path) { +pr_debug ("can't get tracing/saved_cmdline" ); +return -ENOMEM ; + } + + ret =stat (path, &st); +if (ret <0 ) { +/* not found */ + size =0 ; +if (write (output_fd, &size,8 ) !=8 ) + err = -EIO ; + goto out; + } + err =record_file (path,8 ); + +out: +put_tracing_file (path); +return err; +} +static void put_tracepoints_path (struct tracepoint_path *tps) {@@ -546,6 +574,9 @@ struct tracing_data *tracing_data_get(struct list_head *pattrs,if (err) goto out; err =record_ftrace_printk (); +if (err) + goto out; + err =record_saved_cmdline (); out:/* tools/perf/util/trace-event-parse.c +0
-0
tools/perf/util/trace-event-parse.c void parse_ftrace_printk(struct pevent *pevent, · line 236
Quoted 5 lines
tools/perf/util/trace-event-parse.c +2
-0@@ -232,6 +232,23 @@ void parse_ftrace_printk(struct pevent *pevent, } } +void parse_saved_cmdline (struct pevent *pevent, +char *file,unsigned int size__maybe_unused )
any reason for the size parameter? I can see some other functions already following this prototype, but I dont see the reason.. just curious ;)
void parse_ftrace_printk(struct pevent *pevent,
char *file, unsigned int size __maybe_unused)
void parse_proc_kallsyms(struct pevent *pevent,
char *file, unsigned int size __maybe_unused)
jirka
Namhyung Kim
maintainerRe: [PATCH 03/15] perf util: Save pid-cmdline mapping into tracing header
Quoted 7 lines · Jiri Olsa
tools/perf/util/trace-event-parse.c void parse_ftrace_printk(struct pevent *pevent, · line 236
Quoted 2 lines · Namhyung Kim
tools/perf/util/trace-event-parse.c +2
-0+void parse_saved_cmdline (struct pevent *pevent, +char *file,unsigned int size__maybe_unused )any reason for the size parameter? I can see some other functions already following this prototype, but I dont see the reason.. just curious ;)
I don't know, but copied those. ;) I guess we can get rid of it. Steve, any comments?
Thanks,
Namhyung
Quoted 9 lines
void parse_ftrace_printk(struct pevent *pevent,
char *file, unsigned int size__maybe_unused)void parse_proc_kallsyms(struct pevent *pevent,
char *file, unsigned int size__maybe_unused)jirka
Namhyung Kim
maintainerPATCH 4/15 perf util: Add more debug message on failure path
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
It's helpful for debugging on tracing features.
--- tools/perf/util/header.c | 4 ++- tools/perf/util/trace-event-read.c | 53 ++++++++++++++++++++++++++------------ 2 files changed, 39 insertions(+), 18 deletions(-)
tools/perf/util/header.c +3 -1
@@ -2756,8 +2756,10 @@ static int perf_evsel__prepare_tracepoint_event(struct perf_evsel *evsel, } event =pevent_find_event (pevent, evsel->attr.config); -if (event ==NULL ) +if (event ==NULL ) { +pr_debug ("cannot find event format for %d\n" , (int )evsel->attr.config);return -1 ; + }if (!evsel->name) {snprintf (bf,sizeof (bf),"%s:%s" , event->system, event->name);
tools/perf/util/trace-event-read.c +36 -17
99 lines not on this page · raw message
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 5/15 perf tools: Introduce new 'ftrace' tool
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
The ftrace command is a simple wrapper of kernel's ftrace functionality. It only supports single thread tracing currently and just reads trace_pipe in text and then write it to stdout.
--- tools/perf/Makefile | 1 + tools/perf/builtin-ftrace.c | 227 ++++++++++++++++++++++++++++++++++++++++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/perf.c | 1 + 5 files changed, 231 insertions(+) create mode 100644 tools/perf/builtin-ftrace.c
tools/perf/Makefile +1 -0
@@ -552,6 +552,7 @@ BUILTIN_OBJS += $(OUTPUT)builtin-kmem.o BUILTIN_OBJS += $(OUTPUT)builtin-lock.o BUILTIN_OBJS += $(OUTPUT)builtin-kvm.o BUILTIN_OBJS += $(OUTPUT)builtin-inject.o +BUILTIN_OBJS += $(OUTPUT)builtin-ftrace.o BUILTIN_OBJS += $(OUTPUT)tests/builtin-test.o BUILTIN_OBJS += $(OUTPUT)builtin-mem.o
tools/perf/builtin-ftrace.c new file +227 -0
227 lines not on this page · raw message
tools/perf/builtin.h +1 -0
@@ -37,6 +37,7 @@ extern int cmd_test(int argc, const char **argv, const char *prefix);extern int cmd_trace (int argc,const char **argv,const char *prefix);extern int cmd_inject (int argc,const char **argv,const char *prefix);extern int cmd_mem (int argc,const char **argv,const char *prefix); +extern int cmd_ftrace (int argc,const char **argv,const char *prefix);extern int find_scripts (char **scripts_array,char **scripts_path_array);#endif
tools/perf/command-list.txt +1 -0
@@ -9,6 +9,7 @@ perf-buildid-cache mainporcelain common perf-buildid-list mainporcelain common perf-diff mainporcelain common perf-evlist mainporcelain common +perf-ftrace mainporcelain common perf-inject mainporcelain common perf-kmem mainporcelain common perf-kvm mainporcelain common
tools/perf/perf.c +1 -0
@@ -61,6 +61,7 @@ static struct cmd_struct commands[] = {#endif { "inject", cmd_inject, 0 }, { "mem", cmd_mem, 0 }, + { "ftrace", cmd_ftrace, 0 }, }; struct pager_config {
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 6/15 perf ftrace: Add support for --pid option
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
The -p (--pid) option enables to trace existing process by its pid.
--- tools/perf/builtin-ftrace.c | 89 ++++++++++++++++++++++++++++++++------------- 1 file changed, 63 insertions(+), 26 deletions(-)
tools/perf/builtin-ftrace.c +63 -26
@@ -11,6 +11,7 @@#include <unistd.h> #include <signal.h> +#include <fcntl.h> #include "util/debug.h" #include "util/parse-options.h"
@@ -34,11 +35,12 @@ static void sig_handler(int sig __maybe_unused) done = true; } -static int write_tracing_file (const char *name,const char *val) +static int __write_tracing_file (const char *name,const char *val,bool append) {char *file;int fd, ret =-1 ;ssize_t size =strlen (val); +int flags =O_WRONLY ; file =get_tracing_file (name);if (!file) {
@@ -46,7 +48,12 @@ static int write_tracing_file(const char *name, const char *val)return -1 ; } - fd =open (file,O_WRONLY ); +if (append) + flags |=O_APPEND ; +else + flags |=O_TRUNC ; + + fd =open (file, flags);if (fd <0 ) {pr_debug ("cannot open tracing file: %s\n" , name); goto out;
@@ -62,6 +69,16 @@ out:return ret; } +static int write_tracing_file (const char *name,const char *val) +{ +return __write_tracing_file (name, val, false); +} + +static int append_tracing_file (const char *name,const char *val) +{ +return __write_tracing_file (name, val, true); +} +static int reset_tracing_files (struct perf_ftrace *ftrace__maybe_unused ) {if (write_tracing_file ("tracing_on" ,"0" ) <0 )
@@ -76,11 +93,27 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)return 0 ; } +static int set_tracing_pid (struct perf_ftrace *ftrace) +{ +int i; +char buf[16 ]; + +if (perf_target__has_cpu (&ftrace->target)) +return 0 ; + +for (i =0 ; i <thread_map__nr (ftrace->evlist->threads); i++) { +scnprintf (buf,sizeof (buf),"%d" , + ftrace->evlist->threads->map[i]); +if (append_tracing_file ("set_ftrace_pid" , buf) <0 ) +return -1 ; + } +return 0 ; +} +static int __cmd_ftrace (struct perf_ftrace *ftrace,int argc,const char **argv) {char *trace_file;int trace_fd; -char *trace_pid;char buf[4096 ];struct pollfd pollfd = { .events =POLLIN ,
@@ -91,42 +124,37 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)return -1 ; } -if (argc <1 ) -return -1 ; -signal (SIGINT , sig_handler);signal (SIGUSR1 , sig_handler);signal (SIGCHLD , sig_handler); -reset_tracing_files (ftrace); +if (reset_tracing_files (ftrace) <0 ) + goto out;/* reset ftrace buffer */ if (write_tracing_file ("trace" ,"0" ) <0 ) goto out; -if (perf_evlist__prepare_workload (ftrace->evlist, &ftrace->target, - argv, false, true) <0 ) - goto out; - -if (write_tracing_file ("current_tracer" , ftrace->tracer) <0 ) { -pr_err ("failed to set current_tracer to %s\n" , ftrace->tracer); +if (argc &&perf_evlist__prepare_workload (ftrace->evlist, + &ftrace->target, + argv, false, true) <0 ) { goto out; } -if (asprintf (&trace_pid,"%d" , ftrace->evlist->threads->map[0 ]) <0 ) { -pr_err ("failed to allocate pid string\n" ); - goto out; +if (set_tracing_pid (ftrace) <0 ) { +pr_err ("failed to set ftrace pid\n" ); + goto out_reset; } -if (write_tracing_file ("set_ftrace_pid" , trace_pid) <0 ) { -pr_err ("failed to set pid: %s\n" , trace_pid); - goto out_free_pid; +if (write_tracing_file ("current_tracer" , ftrace->tracer) <0 ) { +pr_err ("failed to set current_tracer to %s\n" , ftrace->tracer); + goto out_reset; } trace_file =get_tracing_file ("trace_pipe" );if (!trace_file) {pr_err ("failed to open trace_pipe\n" ); - goto out_free_pid; + goto out_reset; } trace_fd =open (trace_file,O_RDONLY );
@@ -135,7 +163,7 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)if (trace_fd <0 ) {pr_err ("failed to open trace_pipe\n" ); - goto out_free_pid; + goto out_reset; }fcntl (trace_fd,F_SETFL ,O_NONBLOCK );
@@ -174,11 +202,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv) out_close_fd:close (trace_fd); -out_free_pid: -free (trace_pid); -out: +out_reset:reset_tracing_files (ftrace); - +out:return done ?0 :-1 ; }
@@ -196,6 +222,8 @@ int cmd_ftrace(int argc, const char **argv, const char *prefix __maybe_unused)const struct option ftrace_options[] = {OPT_STRING ('t' ,"tracer" , &ftrace.tracer,"tracer" ,"tracer to use" ), +OPT_STRING ('p' ,"pid" , &ftrace.target.tid,"pid" , +"trace on existing process id" ),OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose" ),OPT_END ()
@@ -203,9 +231,18 @@ int cmd_ftrace(int argc, const char **argv, const char *prefix __maybe_unused) argc =parse_options (argc, argv, ftrace_options, ftrace_usage,PARSE_OPT_STOP_AT_NON_OPTION ); -if (!argc) +if (!argc &&perf_target__none (&ftrace.target))usage_with_options (ftrace_usage, ftrace_options); + ret =perf_target__validate (&ftrace.target); +if (ret) { +char errbuf[512 ]; + +perf_target__strerror (&ftrace.target, ret, errbuf,512 ); +pr_err ("%s\n" , errbuf); +return -EINVAL ; + } + ftrace.evlist =perf_evlist__new ();if (ftrace.evlist ==NULL )return -ENOMEM ;
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 7/15 perf ftrace: Add support for -a and -C option
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
The -a/--all-cpus and -C/--cpu option is for controlling tracing cpus. To do that, add and use cpu_map__sprintf() function.
--- tools/perf/builtin-ftrace.c | 69 +++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/cpumap.c | 45 +++++++++++++++++++++++++++++ tools/perf/util/cpumap.h | 1 + 3 files changed, 115 insertions(+)
tools/perf/builtin-ftrace.c +69 -0
@@ -18,6 +18,7 @@#include "util/evlist.h" #include "util/target.h" #include "util/thread_map.h" +#include "util/cpumap.h" #define DEFAULT_TRACER "function_graph"
@@ -79,6 +80,8 @@ static int append_tracing_file(const char *name, const char *val)return __write_tracing_file (name, val, true); } +static int reset_tracing_cpu (void ); +static int reset_tracing_files (struct perf_ftrace *ftrace__maybe_unused ) {if (write_tracing_file ("tracing_on" ,"0" ) <0 )
@@ -90,6 +93,9 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)if (write_tracing_file ("set_ftrace_pid" ," " ) <0 )return -1 ; +if (reset_tracing_cpu () <0 ) +return -1 ; +return 0 ; }
@@ -110,6 +116,60 @@ static int set_tracing_pid(struct perf_ftrace *ftrace)return 0 ; } +static int set_tracing_cpu (struct perf_ftrace *ftrace) +{ +char *cpumask; +size_t mask_size; +int ret; +int last_cpu; +struct cpu_map *cpumap = ftrace->evlist->cpus; + +if (!perf_target__has_cpu (&ftrace->target)) +return 0 ; + + last_cpu = cpumap->map[cpumap->nr -1 ]; + mask_size = (last_cpu +3 ) /4 +1 ; + mask_size += last_cpu /32 ;/* ',' is needed for every 32th cpus */ + + cpumask =malloc (mask_size); +if (cpumask ==NULL ) { +pr_debug ("failed to allocate cpu mask\n" ); +return -1 ; + } + +cpu_map__sprintf (cpumap, cpumask); + + ret =write_tracing_file ("tracing_cpumask" , cpumask); + +free (cpumask); +return ret; +} + +static int reset_tracing_cpu (void ) +{ +char *cpumask; +size_t mask_size; +int last_cpu; +struct cpu_map *cpumap =cpu_map__new (NULL ); + + last_cpu = cpumap->map[cpumap->nr -1 ]; + mask_size = (last_cpu +3 ) /4 +1 ; + mask_size += last_cpu /32 ;/* ',' is needed for every 32th cpus */ + + cpumask =malloc (mask_size); +if (cpumask ==NULL ) { +pr_debug ("failed to allocate cpu mask\n" ); +return -1 ; + } + +cpu_map__sprintf (cpumap, cpumask); + +write_tracing_file ("tracing_cpumask" , cpumask); + +free (cpumask); +return 0 ; +} +static int __cmd_ftrace (struct perf_ftrace *ftrace,int argc,const char **argv) {char *trace_file;
@@ -146,6 +206,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv) goto out_reset; } +if (set_tracing_cpu (ftrace) <0 ) { +pr_err ("failed to set tracing cpumask\n" ); + goto out_reset; + } +if (write_tracing_file ("current_tracer" , ftrace->tracer) <0 ) {pr_err ("failed to set current_tracer to %s\n" , ftrace->tracer); goto out_reset;
@@ -226,6 +291,10 @@ int cmd_ftrace(int argc, const char **argv, const char *prefix __maybe_unused) "trace on existing process id"),OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose" ), +OPT_BOOLEAN ('a' ,"all-cpus" , &ftrace.target.system_wide, +"system-wide collection from all CPUs" ), +OPT_STRING ('C' ,"cpu" , &ftrace.target.cpu_list,"cpu" , +"list of cpus to monitor" ), OPT_END() };
tools/perf/util/cpumap.c +45 -0
51 lines not on this page · raw message
tools/perf/util/cpumap.h +1 -0
@@ -14,6 +14,7 @@ struct cpu_map *cpu_map__dummy_new(void);void cpu_map__delete (struct cpu_map *map);struct cpu_map *cpu_map__read (FILE *file);size_t cpu_map__fprintf (struct cpu_map *map,FILE *fp); +size_t cpu_map__sprintf (struct cpu_map *map,char *buf);int cpu_map__get_socket (struct cpu_map *map,int idx);int cpu_map__get_core (struct cpu_map *map,int idx);int cpu_map__build_socket_map (struct cpu_map *cpus,struct cpu_map **sockp);
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 8/15 perf ftrace: Split "live" sub-command
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
Separate out the default behavior to "live" subcommand. It's a preparation to support more subcommands like "record" and "report".
--- tools/perf/builtin-ftrace.c | 109 +++++++++++++++++++++++++++----------------- 1 file changed, 68 insertions(+), 41 deletions(-)
tools/perf/builtin-ftrace.c +68 -41
@@ -170,7 +170,7 @@ static int reset_tracing_cpu(void)return 0 ; } -static int __cmd_ftrace (struct perf_ftrace *ftrace,int argc,const char **argv) +static int do_ftrace_live (struct perf_ftrace *ftrace) {char *trace_file;int trace_fd;
@@ -179,11 +179,6 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv) .events = POLLIN, }; -if (geteuid () !=0 ) { -pr_err ("ftrace only works for root!\n" ); -return -1 ; - } -signal (SIGINT , sig_handler);signal (SIGUSR1 , sig_handler);signal (SIGCHLD , sig_handler);
@@ -195,12 +190,6 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)if (write_tracing_file ("trace" ,"0" ) <0 ) goto out; -if (argc &&perf_evlist__prepare_workload (ftrace->evlist, - &ftrace->target, - argv, false, true) <0 ) { - goto out; - } -if (set_tracing_pid (ftrace) <0 ) {pr_err ("failed to set ftrace pid\n" ); goto out_reset;
@@ -273,61 +262,99 @@ out:return done ?0 :-1 ; } -int cmd_ftrace (int argc,const char **argv,const char *prefix__maybe_unused ) +static int +__cmd_ftrace_live (struct perf_ftrace *ftrace,int argc,const char **argv) { -int ret; -struct perf_ftrace ftrace = { - .target = { .uid =UINT_MAX , }, - }; -const char *const ftrace_usage[] = { -"perf ftrace [<options>] [<command>]" , -"perf ftrace [<options>] -- <command> [<options>]" , +int ret =-1 ; +const char *const live_usage[] = { +"perf ftrace live [<options>] [<command>]" , +"perf ftrace live [<options>] -- <command> [<options>]" ,NULL }; -const struct option ftrace_options[] = { -OPT_STRING ('t' ,"tracer" , &ftrace.tracer,"tracer" , +const struct option live_options[] = { +OPT_STRING ('t' ,"tracer" , &ftrace->tracer,"tracer" ,"tracer to use" ), -OPT_STRING ('p' ,"pid" , &ftrace.target.tid,"pid" , +OPT_STRING ('p' ,"pid" , &ftrace->target.tid,"pid" ,"trace on existing process id" ),OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose" ), -OPT_BOOLEAN ('a' ,"all-cpus" , &ftrace.target.system_wide, +OPT_BOOLEAN ('a' ,"all-cpus" , &ftrace->target.system_wide,"system-wide collection from all CPUs" ), -OPT_STRING ('C' ,"cpu" , &ftrace.target.cpu_list,"cpu" , +OPT_STRING ('C' ,"cpu" , &ftrace->target.cpu_list,"cpu" ,"list of cpus to monitor" ),OPT_END () }; - argc =parse_options (argc, argv, ftrace_options, ftrace_usage, -PARSE_OPT_STOP_AT_NON_OPTION ); -if (!argc &&perf_target__none (&ftrace.target)) -usage_with_options (ftrace_usage, ftrace_options); + argc =parse_options (argc, argv, live_options, live_usage, +PARSE_OPT_STOP_AT_NON_OPTION ); +if (!argc &&perf_target__none (&ftrace->target)) +usage_with_options (live_usage, live_options); - ret =perf_target__validate (&ftrace.target); + ret =perf_target__validate (&ftrace->target);if (ret) {char errbuf[512 ]; -perf_target__strerror (&ftrace.target, ret, errbuf,512 ); +perf_target__strerror (&ftrace->target, ret, errbuf,512 );pr_err ("%s\n" , errbuf);return -EINVAL ; } - ftrace.evlist =perf_evlist__new (); -if (ftrace.evlist ==NULL ) + ftrace->evlist =perf_evlist__new (); +if (ftrace->evlist ==NULL )return -ENOMEM ; - ret =perf_evlist__create_maps (ftrace.evlist, &ftrace.target); + ret =perf_evlist__create_maps (ftrace->evlist, &ftrace->target);if (ret <0 ) - goto out_delete_evlist; + goto out; -if (ftrace.tracer ==NULL ) - ftrace.tracer =DEFAULT_TRACER ; +if (ftrace->tracer ==NULL ) + ftrace->tracer =DEFAULT_TRACER ; - ret =__cmd_ftrace (&ftrace, argc, argv); +if (argc &&perf_evlist__prepare_workload (ftrace->evlist, + &ftrace->target, + argv, false, true) <0 ) + goto out_maps; + + ret =do_ftrace_live (ftrace); -perf_evlist__delete_maps (ftrace.evlist); -out_delete_evlist: -perf_evlist__delete (ftrace.evlist); +out_maps: +perf_evlist__delete_maps (ftrace->evlist); +out: +perf_evlist__delete (ftrace->evlist); + +return ret; +} + +int cmd_ftrace (int argc,const char **argv,const char *prefix__maybe_unused ) +{ +int ret; +struct perf_ftrace ftrace = { + .target = { .uid =UINT_MAX , }, + }; +const char *const ftrace_usage[] = { +"perf ftrace {live} [<options>] [<command>]" , +"perf ftrace {live} [<options>] -- <command> [<options>]" , +NULL + }; +const struct option ftrace_options[] = { +OPT_END () + }; + + argc =parse_options (argc, argv, ftrace_options, ftrace_usage, +PARSE_OPT_STOP_AT_NON_OPTION ); +if (!argc) +usage_with_options (ftrace_usage, ftrace_options); + +if (geteuid () !=0 ) { +pr_err ("ftrace only works for root!\n" ); +return -1 ; + } + +if (strcmp (argv[0 ],"live" ) ==0 ) { + ret =__cmd_ftrace_live (&ftrace, argc, argv); + }else { +usage_with_options (ftrace_usage, ftrace_options); + }return ret; }
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 9/15 perf ftrace: Add 'record' sub-command
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
The ftrace record command is for saving raw ftrace buffer contents which can be get from per_cpu/cpuX/trace_pipe_raw.
Since ftrace events are generated very frequently so single thread for recording mostly resulted in buffer overruns. Thus it uses per-cpu recorder thread to prevent such cases and they save the contents to their own files.
These per-cpu data files are saved in a directory so that they can be easily found when needed. I chose the default directory name as "perf.data.dir" and the first two (i.e. "perf.data") can be changed with -o option. The structure of the directory looks like:
$ tree perf.data.dir perf.data.dir/ |-- perf.header |-- trace-cpu0.buf |-- trace-cpu1.buf |-- trace-cpu2.buf `-- trace-cpu3.buf
In addition to trace-cpuX.buf files, it has perf.header file also. The perf.header file is compatible with existing perf.data format and contains usual event information, feature mask and sample data. The sample data is synthesized to indicate given cpu has a record file.
--- tools/perf/builtin-ftrace.c | 629 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 611 insertions(+), 18 deletions(-)
tools/perf/builtin-ftrace.c +611 -18
@@ -12,6 +12,7 @@#include <unistd.h> #include <signal.h> #include <fcntl.h> +#include <dirent.h> #include "util/debug.h" #include "util/parse-options.h"
@@ -19,14 +20,17 @@#include "util/target.h" #include "util/thread_map.h" #include "util/cpumap.h" +#include "util/trace-event.h" #define DEFAULT_TRACER "function_graph" +#define DEFAULT_DIRNAME "perf.data"struct perf_ftrace {struct perf_evlist *evlist;struct perf_target target;const char *tracer; +const char *dirname; };static bool done;
@@ -170,41 +174,57 @@ static int reset_tracing_cpu(void)return 0 ; } -static int do_ftrace_live (struct perf_ftrace *ftrace) +static int setup_tracing_files (struct perf_ftrace *ftrace) { -char *trace_file; -int trace_fd; -char buf[4096 ]; -struct pollfd pollfd = { - .events =POLLIN , - }; - -signal (SIGINT , sig_handler); -signal (SIGUSR1 , sig_handler); -signal (SIGCHLD , sig_handler); +int ret =-1 ; -if (reset_tracing_files (ftrace) <0 ) +if (reset_tracing_files (ftrace) <0 ) { +pr_err ("failed to reset tracing files\n" ); goto out; + }/* reset ftrace buffer */ -if (write_tracing_file ("trace" ,"0" ) <0 ) +if (write_tracing_file ("trace" ,"0" ) <0 ) { +pr_err ("failed to reset ftrace buffer\n" ); goto out; + }if (set_tracing_pid (ftrace) <0 ) {pr_err ("failed to set ftrace pid\n" ); - goto out_reset; + goto out; }if (set_tracing_cpu (ftrace) <0 ) {pr_err ("failed to set tracing cpumask\n" ); - goto out_reset; + goto out; }if (write_tracing_file ("current_tracer" , ftrace->tracer) <0 ) {pr_err ("failed to set current_tracer to %s\n" , ftrace->tracer); - goto out_reset; + goto out; } + ret =0 ; +out: +return ret; +} + +static int do_ftrace_live (struct perf_ftrace *ftrace) +{ +char *trace_file; +int trace_fd; +char buf[4096 ]; +struct pollfd pollfd = { + .events =POLLIN , + }; + +signal (SIGINT , sig_handler); +signal (SIGUSR1 , sig_handler); +signal (SIGCHLD , sig_handler); + +if (setup_tracing_files (ftrace) <0 ) + goto out_reset; + trace_file =get_tracing_file ("trace_pipe" );if (!trace_file) {pr_err ("failed to open trace_pipe\n" );
@@ -258,7 +278,510 @@ out_close_fd:close (trace_fd); out_reset:reset_tracing_files (ftrace); +return done ?0 :-1 ; +} + +static int alloc_ftrace_evsel (struct perf_ftrace *ftrace) +{ +if (!strcmp (ftrace->tracer,"function" )) { +if (perf_evlist__add_newtp (ftrace->evlist,"ftrace" , +"function" ,NULL ) <0 ) { +pr_err ("failed to allocate ftrace event\n" ); +return -1 ; + } + }else if (!strcmp (ftrace->tracer,"function_graph" )) { +if (perf_evlist__add_newtp (ftrace->evlist,"ftrace" , +"funcgraph_entry" ,NULL ) || +perf_evlist__add_newtp (ftrace->evlist,"ftrace" , +"funcgraph_exit" ,NULL )) { +pr_err ("failed to allocate ftrace event\n" ); +return -1 ; + } + }else { +pr_err ("Not supported tracer: %s\n" , ftrace->tracer); +return -1 ; + } +return 0 ; +} + +static int remove_directory (const char *pathname) +{ +DIR *dir; +int ret =0 ; +struct dirent *dent; +char namebuf[PATH_MAX ]; + + dir =opendir (pathname); +if (dir ==NULL ) +return 0 ; + +while ((dent =readdir (dir)) !=NULL && !ret) { +struct stat statbuf; + +if (dent->d_name[0 ] =='.' ) +continue ; + +scnprintf (namebuf,sizeof (namebuf),"%s/%s" , + pathname, dent->d_name); + + ret =stat (namebuf, &statbuf); +if (ret <0 ) { +pr_debug ("stat failed\n" ); +break ; + } + +if (S_ISREG (statbuf.st_mode)) + ret =unlink (namebuf); +else if (S_ISDIR (statbuf.st_mode)) + ret =remove_directory (namebuf); +else { +pr_debug ("unknown file.\n" ); + ret =-1 ; + } + } +closedir (dir); + +if (ret <0 ) +return ret; + +return rmdir (pathname); +} + +static int create_perf_directory (struct perf_ftrace *ftrace) +{ +int err; +char buf[PATH_MAX ]; +struct stat statbuf; + +scnprintf (buf,sizeof (buf),"%s.dir" , ftrace->dirname); + +if (stat (buf, &statbuf) ==0 ) { +/* same name already exists - rename to *.old.dir */ +char *old_name =malloc (strlen (buf) +5 ); +if (old_name ==NULL ) +return -1 ; + +scnprintf (old_name,strlen (buf) +5 , +"%s.old.dir" , ftrace->dirname); + +if (remove_directory (old_name) <0 ) { +perror ("rmdir" ); +return -1 ; + } + +if (rename (buf, old_name) <0 ) { +perror ("rename" ); +free (old_name); +return -1 ; + } + +free (old_name); + } + + err =mkdir (buf,0755 ); +if (err <0 ) { +perror ("mkdir" ); +return -1 ; + } + +strcat (buf,"/perf.header" ); + + err =open (buf,O_RDWR |O_CREAT |O_TRUNC ,0644 ); +return err; +} + +static void sig_dummy_handler (int sig__maybe_unused ) +{ +while (!done) +continue ; +} + +enum { +RECORD_STATE__ERROR =-1 , +RECORD_STATE__INIT , +RECORD_STATE__READY , +RECORD_STATE__DONE , +}; + +struct ftrace_record_arg { +struct perf_ftrace *ftrace; +int cpu; +int state; +pthread_t id; +struct list_head node; +}; + +static int recorder_count; +pthread_cond_t recorder_ready_cond =PTHREAD_COND_INITIALIZER ; +pthread_cond_t recorder_start_cond =PTHREAD_COND_INITIALIZER ; +pthread_mutex_t recorder_mutex =PTHREAD_MUTEX_INITIALIZER ; + +static void *record_ftrace_raw_buffer (void *arg) +{ +struct ftrace_record_arg *fra = arg; +char buf[4096 ]; +char *trace_file; +int trace_fd; +int output_fd; +int ret, n; +off_t byte_written =0 ; +sigset_t sigmask; +struct pollfd pollfd = { + .events =POLLIN , + }; + + fra->state =RECORD_STATE__ERROR ; + +snprintf (buf,sizeof (buf),"per_cpu/cpu%d/trace_pipe_raw" , fra->cpu); + + trace_file =get_tracing_file (buf); +if (!trace_file) { +pr_err ("failed to get trace_pipe_raw\n" ); + goto out; + } + + trace_fd =open (trace_file,O_RDONLY ); + +put_tracing_file (trace_file); + +if (trace_fd <0 ) { +pr_err ("failed to open trace_pipe_raw\n" ); + goto out; + } + +snprintf (buf,sizeof (buf),"%s.dir/trace-cpu%d.buf" , + fra->ftrace->dirname, fra->cpu); + + output_fd =open (buf,O_WRONLY |O_CREAT |O_TRUNC ,0644 ); +if (output_fd <0 ) { +pr_err ("failed to open output file\n" ); + goto out_close; + } + + fra->state =RECORD_STATE__READY ; + +/* +* block all signals but SIGUSR2. +* It'll be used to unblock a recorder to finish. +*/ +sigfillset (&sigmask); +sigdelset (&sigmask,SIGUSR2 ); +pthread_sigmask (SIG_SETMASK , &sigmask,NULL ); + +signal (SIGUSR2 , sig_dummy_handler); + +fcntl (trace_fd,F_SETFL ,O_NONBLOCK ); + pollfd.fd = trace_fd; + +/* Now I'm ready */ +pthread_mutex_lock (&recorder_mutex); + recorder_count++; +pthread_cond_signal (&recorder_ready_cond); +pthread_cond_wait (&recorder_start_cond, &recorder_mutex); +pthread_mutex_unlock (&recorder_mutex); + +pr_debug2 ("now recording for cpu%d\n" , fra->cpu); + +while (!done) { + ret =poll (&pollfd,1 ,-1 ); +if (ret <0 && errno !=EINTR ) + goto out_close; + +if (pollfd.revents &POLLIN ) { + n =read (trace_fd, buf,sizeof (buf)); +if (n <0 ) + goto out_close; +if (write (output_fd, buf, n) != n) + goto out_close; + byte_written += n; + } + } + +/* read remaining buffer contents */ +while (true) { + n =read (trace_fd, buf,sizeof (buf)); +if (n <0 ) + goto out_close; +if (n ==0 ) +break ; +if (write (output_fd, buf, n) != n) + goto out_close; + byte_written += n; + } + fra->state =RECORD_STATE__DONE ; + +out_close: +close (trace_fd); out: +if (fra->state ==RECORD_STATE__ERROR ) { +/* +* We need to update recorder_count in this case also +* in order to prevent deadlocking in the main thread. +*/ +pthread_mutex_lock (&recorder_mutex); + recorder_count++; +pthread_cond_signal (&recorder_ready_cond); +pthread_mutex_unlock (&recorder_mutex); + } +return fra; +} + +static void *synthesize_raw_data (struct perf_evsel *evsel) +{ +void *data =NULL ; +u32 data_size; + +if (!strcmp (evsel->tp_format->name,"function" )) { +struct { +unsigned short common_type; +unsigned char common_flags; +unsigned char common_preempt_count; +int common_pid; +int common_padding; + +unsigned long ip; +unsigned long parent_ip; + } function_format = { + .common_type = evsel->attr.config, + }; + + data_size =sizeof (function_format); + + data =malloc (data_size +sizeof (u32)); +if (data ==NULL ) +return NULL ; + +memcpy (data, &data_size,sizeof (data_size)); + + data +=sizeof (data_size); +memcpy (data, &function_format,sizeof (function_format)); + }else if (!strcmp (evsel->tp_format->name,"funcgraph_entry" )) { +struct { +unsigned short common_type; +unsigned char common_flags; +unsigned char common_preempt_count; +int common_pid; +int common_padding; + +unsigned long func; +int depth; + } funcgraph_entry_format = { + .common_type = evsel->attr.config, + }; + + data_size =sizeof (funcgraph_entry_format); + + data =malloc (data_size +sizeof (u32)); +if (data ==NULL ) +return NULL ; + +memcpy (data, &data_size,sizeof (data_size)); + + data +=sizeof (data_size); +memcpy (data, &funcgraph_entry_format, +sizeof (funcgraph_entry_format)); + } +return data; +} + +static int do_ftrace_record (struct perf_ftrace *ftrace) +{ +int i, err, feat; +int perf_fd; +LIST_HEAD (recorders); +struct perf_session *session; +struct ftrace_record_arg *fra, *tmp; + +signal (SIGINT , sig_handler); +signal (SIGUSR1 , sig_handler); +signal (SIGCHLD , sig_handler); + +if (setup_tracing_files (ftrace) <0 ) + goto out_reset; + +alloc_ftrace_evsel (ftrace); + + perf_fd =create_perf_directory (ftrace); +if (perf_fd <0 ) { +pr_err ("failed to create perf directory\n" ); + goto out_reset; + } + +/* just use a dummy session for header recording */ + session =zalloc (sizeof (*session)); +if (session ==NULL ) { +pr_err ("failed to allocate perf session\n" ); + goto out_close; + } + session->evlist = ftrace->evlist; + +for (feat =HEADER_FIRST_FEATURE ; feat <HEADER_LAST_FEATURE ; feat++) +perf_header__set_feat (&session->header, feat); + +perf_header__clear_feat (&session->header,HEADER_BUILD_ID ); +perf_header__clear_feat (&session->header,HEADER_BRANCH_STACK ); +perf_header__clear_feat (&session->header,HEADER_PMU_MAPPINGS ); +perf_header__clear_feat (&session->header,HEADER_GROUP_DESC ); + + err =perf_session__write_header (session, ftrace->evlist, + perf_fd, false); +if (err <0 ) { +pr_err ("failed to write perf header\n" ); + goto out_session; + } + +/* +* We record ftrace's per_cpu buffer so that it'd better having +* corresponding cpu maps anyway. +*/ +if (!perf_target__has_cpu (&ftrace->target)) { +struct cpu_map *new_map; + + new_map =cpu_map__new (NULL ); +if (new_map ==NULL ) { +pr_err ("failed to create new cpu map\n" ); + goto out_session; + } + +/* replace existing cpu map */ +cpu_map__delete (ftrace->evlist->cpus); + ftrace->evlist->cpus = new_map; + } + +for (i =0 ; i <cpu_map__nr (ftrace->evlist->cpus); i++) { + fra =malloc (sizeof (*fra)); +if (fra ==NULL ) { +pr_err ("not enough memory!\n" ); + goto out_fra; + } + + fra->ftrace = ftrace; + fra->cpu = ftrace->evlist->cpus->map[i]; + fra->state =RECORD_STATE__INIT ; +list_add_tail (&fra->node, &recorders); + + err =pthread_create (&fra->id,NULL , + record_ftrace_raw_buffer, fra); +if (err <0 ) { +pr_err ("failed to create recorder thread\n" ); + goto out_fra; + } + } + +/* wait for all recorders ready */ +pthread_mutex_lock (&recorder_mutex); +while (recorder_count !=cpu_map__nr (ftrace->evlist->cpus)) +pthread_cond_wait (&recorder_ready_cond, &recorder_mutex); +pthread_mutex_unlock (&recorder_mutex); + +list_for_each_entry (fra, &recorders, node) { +if (fra->state !=RECORD_STATE__READY ) { +pr_err ("cpu%d: failed to start recorder" , fra->cpu); + goto out_fra; + } + } + +if (write_tracing_file ("tracing_on" ,"1" ) <0 ) { +pr_err ("can't enable tracing\n" ); + goto out_fra; + } + +perf_evlist__start_workload (ftrace->evlist); + +pr_debug2 ("start recording per cpu buffers\n" ); +pthread_mutex_lock (&recorder_mutex); +pthread_cond_broadcast (&recorder_start_cond); +pthread_mutex_unlock (&recorder_mutex); + +/* wait for signal/finish */ +pause (); + +if (write_tracing_file ("tracing_on" ,"0" ) <0 ) { +pr_err ("can't disable tracing\n" ); + goto out_fra; + } + +/* signal recorders to terminate */ +list_for_each_entry (fra, &recorders, node) { +pr_debug2 ("killing recorder thread for cpu%d\n" , fra->cpu); +pthread_kill (fra->id,SIGUSR2 ); + } + +list_for_each_entry (fra, &recorders, node) +pthread_join (fra->id,NULL ); + +/* synthesize sample data */ +list_for_each_entry (fra, &recorders, node) { +struct perf_evsel *evsel =perf_evlist__first (ftrace->evlist); +union perf_event event = { + .sample = { + .header = { + .type =PERF_RECORD_SAMPLE , + .misc =PERF_RECORD_MISC_KERNEL , + .size =sizeof (event.sample.header) + + evsel->sample_size, + }, + }, + }; +struct perf_sample sample = { + .cpu = fra->cpu, + .period =1 , + }; +void *raw_data; +u32 raw_size; +int orig_size; + +if (fra->state !=RECORD_STATE__DONE ) { +pr_warning ("recorder failed for some reason on cpu%d\n" , + fra->cpu); +continue ; + } + +perf_event__synthesize_sample (&event, evsel->attr.sample_type, + &sample, false); + + raw_data =synthesize_raw_data (evsel); +if (raw_data ==NULL ) { +pr_err ("synthesizing raw sample failed\n" ); + goto out_fra; + } + +/* +* start of raw data is the size of raw data excluding itself. +*/ + raw_size =sizeof (u32) + (*(u32 *) raw_data); + + orig_size = event.sample.header.size; + event.sample.header.size += raw_size; + + err =write (perf_fd, &event.sample, orig_size); +if (err != orig_size) { +pr_err ("write error occurred\n" ); + goto out_fra; + } + + err =write (perf_fd, raw_data, raw_size); +if (err != (int )raw_size) { +pr_err ("write error occurred\n" ); + goto out_fra; + } + + session->header.data_size += event.sample.header.size; + } + +perf_session__write_header (session, ftrace->evlist, perf_fd, true); + +out_fra: +list_for_each_entry_safe (fra, tmp, &recorders, node) { +list_del (&fra->node); +free (fra); + } +out_session: +free (session); +out_close: +close (perf_fd); +out_reset: +reset_tracing_files (ftrace);return done ?0 :-1 ; }
@@ -325,6 +848,74 @@ out:return ret; } +static int +__cmd_ftrace_record (struct perf_ftrace *ftrace,int argc,const char **argv) +{ +int ret =-1 ; +const char *const record_usage[] = { +"perf ftrace record [<options>] [<command>]" , +"perf ftrace record [<options>] -- <command> [<options>]" , +NULL + }; +const struct option record_options[] = { +OPT_STRING ('t' ,"tracer" , &ftrace->tracer,"tracer" , +"tracer to use" ), +OPT_STRING ('p' ,"pid" , &ftrace->target.tid,"pid" , +"trace on existing process id" ), +OPT_INCR ('v' ,"verbose" , &verbose, +"be more verbose" ), +OPT_BOOLEAN ('a' ,"all-cpus" , &ftrace->target.system_wide, +"system-wide collection from all CPUs" ), +OPT_STRING ('C' ,"cpu" , &ftrace->target.cpu_list,"cpu" , +"list of cpus to monitor" ), +OPT_STRING ('o' ,"output" , &ftrace->dirname,"dirname" , +"input directory name to use (default: perf.data)" ), +OPT_END () + }; + + argc =parse_options (argc, argv, record_options, record_usage, +PARSE_OPT_STOP_AT_NON_OPTION ); +if (!argc &&perf_target__none (&ftrace->target)) +usage_with_options (record_usage, record_options); + + ret =perf_target__validate (&ftrace->target); +if (ret) { +char errbuf[512 ]; + +perf_target__strerror (&ftrace->target, ret, errbuf,512 ); +pr_err ("%s\n" , errbuf); +return -EINVAL ; + } + + ftrace->evlist =perf_evlist__new (); +if (ftrace->evlist ==NULL ) +return -ENOMEM ; + + ret =perf_evlist__create_maps (ftrace->evlist, &ftrace->target); +if (ret <0 ) + goto out; + +if (ftrace->tracer ==NULL ) + ftrace->tracer =DEFAULT_TRACER ; + +if (ftrace->dirname ==NULL ) + ftrace->dirname =DEFAULT_DIRNAME ; + +if (argc &&perf_evlist__prepare_workload (ftrace->evlist, + &ftrace->target, + argv, false, true) <0 ) + goto out_maps; + + ret =do_ftrace_record (ftrace); + +out_maps: +perf_evlist__delete_maps (ftrace->evlist); +out: +perf_evlist__delete (ftrace->evlist); + +return ret; +} +int cmd_ftrace (int argc,const char **argv,const char *prefix__maybe_unused ) {int ret;
@@ -332,8 +923,8 @@ int cmd_ftrace(int argc, const char **argv, const char *prefix __maybe_unused) .target = { .uid = UINT_MAX, }, };const char *const ftrace_usage[] = { -"perf ftrace {live} [<options>] [<command>]" , -"perf ftrace {live} [<options>] -- <command> [<options>]" , +"perf ftrace {live|record} [<options>] [<command>]" , +"perf ftrace {live|record} [<options>] -- <command> [<options>]" ,NULL };const struct option ftrace_options[] = {
@@ -352,6 +943,8 @@ int cmd_ftrace(int argc, const char **argv, const char *prefix __maybe_unused)if (strcmp (argv[0 ],"live" ) ==0 ) { ret =__cmd_ftrace_live (&ftrace, argc, argv); + }else if (strncmp (argv[0 ],"rec" ,3 ) ==0 ) { + ret =__cmd_ftrace_record (&ftrace, argc, argv); }else {usage_with_options (ftrace_usage, ftrace_options); }
Signature · 1 line
1.7.11.7
Re: [PATCH 09/15] perf ftrace: Add 'record' sub-command
Quoted 10 lines · Namhyung Kim
From: Namhyung Kim <[email protected]>
The ftrace record command is for saving raw ftrace buffer contents which can be get from per_cpu/cpuX/trace_pipe_raw.
Since ftrace events are generated very frequently so single thread for recording mostly resulted in buffer overruns. Thus it uses per-cpu recorder thread to prevent such cases and they save the contents to their own files.
SNIP
Quoted 46 lines
tools/perf/builtin-ftrace.c +46
-0+ +/* synthesize sample data */ +list_for_each_entry (fra, &recorders, node) { +struct perf_evsel *evsel =perf_evlist__first (ftrace->evlist); +union perf_event event = { + .sample = { + .header = { + .type =PERF_RECORD_SAMPLE , + .misc =PERF_RECORD_MISC_KERNEL , + .size =sizeof (event.sample.header) + + evsel->sample_size, + }, + }, + }; +struct perf_sample sample = { + .cpu = fra->cpu, + .period =1 , + }; +void *raw_data; +u32 raw_size; +int orig_size; + +if (fra->state !=RECORD_STATE__DONE ) { +pr_warning ("recorder failed for some reason on cpu%d\n" , + fra->cpu); +continue ; + } + +perf_event__synthesize_sample (&event, evsel->attr.sample_type, + &sample, false); + + raw_data =synthesize_raw_data (evsel); +if (raw_data ==NULL ) { +pr_err ("synthesizing raw sample failed\n" ); + goto out_fra; + } + +/* +* start of raw data is the size of raw data excluding itself. +*/ + raw_size =sizeof (u32) + (*(u32 *) raw_data); + + orig_size = event.sample.header.size; + event.sample.header.size += raw_size; + + err =write (perf_fd, &event.sample, orig_size);
tools/perf/builtin-ftrace.c out_close_fd: · line 767
Quoted 10 lines
tools/perf/builtin-ftrace.c +10
-0+if (err != orig_size) { +pr_err ("write error occurred\n" ); + goto out_fra; + } + + err =write (perf_fd, raw_data, raw_size); +if (err != (int )raw_size) { +pr_err ("write error occurred\n" ); + goto out_fra; + }
missing free(raw_data) ?
jirka
Quoted 20 lines
tools/perf/builtin-ftrace.c +17
-0+ + session->header.data_size += event.sample.header.size; + } + +perf_session__write_header (session, ftrace->evlist, perf_fd, true); + +out_fra: +list_for_each_entry_safe (fra, tmp, &recorders, node) { +list_del (&fra->node); +free (fra); + } +out_session: +free (session); +out_close: +close (perf_fd); +out_reset: +reset_tracing_files (ftrace);return done ?0 :-1 ; }
Namhyung Kim
maintainerRe: [PATCH 09/15] perf ftrace: Add 'record' sub-command
Quoted 20 lines · Jiri Olsa
Quoted 19 lines · Namhyung Kim
tools/perf/builtin-ftrace.c +19
-0+ raw_data =synthesize_raw_data (evsel); +if (raw_data ==NULL ) { +pr_err ("synthesizing raw sample failed\n" ); + goto out_fra; + } + +/* +* start of raw data is the size of raw data excluding itself. +*/ + raw_size =sizeof (u32) + (*(u32 *) raw_data); + + orig_size = event.sample.header.size; + event.sample.header.size += raw_size; + + err =write (perf_fd, &event.sample, orig_size); +if (err != orig_size) { +pr_err ("write error occurred\n" ); + goto out_fra; + }
Quoted 8 lines
tools/perf/builtin-ftrace.c out_close_fd: · line 767
Quoted 6 lines
tools/perf/builtin-ftrace.c +6
-0+ + err =write (perf_fd, raw_data, raw_size); +if (err != (int )raw_size) { +pr_err ("write error occurred\n" ); + goto out_fra; + }missing
free(raw_data)?
Argh, right!
Thanks,
Namhyung
Re: [PATCH 09/15] perf ftrace: Add 'record' sub-command
In reply to Namhyung KimQuoted 23 lines · Namhyung Kim
From: Namhyung Kim <[email protected]>
The ftrace record command is for saving raw ftrace buffer contents which can be get from per_cpu/cpuX/trace_pipe_raw.
Since ftrace events are generated very frequently so single thread for recording mostly resulted in buffer overruns. Thus it uses per-cpu recorder thread to prevent such cases and they save the contents to their own files.
These per-cpu data files are saved in a directory so that they can be easily found when needed. I chose the default directory name as "perf.data.dir" and the first two (i.e. "perf.data") can be changed with -o option. The structure of the directory looks like:
$ tree perf.data.dir perf.data.dir/ |-- perf.header |-- trace-cpu0.buf |-- trace-cpu1.buf |-- trace-cpu2.buf `-- trace-cpu3.buf
Quoted 4 lines
In addition to trace-cpuX.buf files, it has perf.header file also. The perf.header file is compatible with existing perf.data format and contains usual event information, feature mask and sample data. The sample data is synthesized to indicate given cpu has a record file.
so AFAICS we store sample event for each cpu we recorded (in record command) and by processing those samples (show/report) we get data files names for each cpu
hum, I might have missed some discussion about this, but seems like header feature fits better for this, as long as we store only cpus.. FTRACE_CPUS or something like thius with simple array
or if we are going to use event, maybe user event via enum perf_user_event_type fits better
jirka
Namhyung Kim
maintainerRe: [PATCH 09/15] perf ftrace: Add 'record' sub-command
Quoted 24 lines · Jiri Olsa
Quoted 23 lines · Namhyung Kim
From: Namhyung Kim <[email protected]>
The ftrace record command is for saving raw ftrace buffer contents which can be get from per_cpu/cpuX/trace_pipe_raw.
Since ftrace events are generated very frequently so single thread for recording mostly resulted in buffer overruns. Thus it uses per-cpu recorder thread to prevent such cases and they save the contents to their own files.
These per-cpu data files are saved in a directory so that they can be easily found when needed. I chose the default directory name as "perf.data.dir" and the first two (i.e. "perf.data") can be changed with -o option. The structure of the directory looks like:
$ tree perf.data.dir perf.data.dir/ |-- perf.header |-- trace-cpu0.buf |-- trace-cpu1.buf |-- trace-cpu2.buf `-- trace-cpu3.buf
Quoted 8 lines
Quoted 4 lines
In addition to trace-cpuX.buf files, it has perf.header file also. The perf.header file is compatible with existing perf.data format and contains usual event information, feature mask and sample data. The sample data is synthesized to indicate given cpu has a record file.
so AFAICS we store sample event for each cpu we recorded (in record command) and by processing those samples (show/report) we get data files names for each cpu
Correct.
Quoted 4 lines
hum, I might have missed some discussion about this, but seems like header feature fits better for this, as long as we store only cpus..
FTRACE_CPUSor something like thius with simple array
Well, there's no discussion on this side so far. :)
I wanted not to add a new header feature only for this but use existing method (sample event) so that it also can be recognized - at least to have very rough idea - by an incompatible version of the perf tool. Maybe the event has more information like total number of samples as a period.
Quoted 3 lines
or if we are going to use event, maybe user event via enum
perf_user_event_typefits better
IIRC some people don't want to add more user event types.
Thanks,
Namhyung
Namhyung Kim
maintainerPATCH 10/15 perf ftrace: Add 'show' sub-command
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
The ftrace show subcommand is for viewing recorded ftrace files. It enters perf.data.dir directory and open perf.header file to find out necessary information. And then read out per-cpu trace records using kbuffer helper and print them to stdout in time order.
It only shows its basic form so function graph doesn't show duration/ overhead and no leaf entry handling is provided yet. Maybe it can be handled by a proper plugin in the libtraceevent.
--- tools/perf/builtin-ftrace.c | 427 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 425 insertions(+), 2 deletions(-)
tools/perf/builtin-ftrace.c +425 -2
@@ -13,6 +13,7 @@#include <signal.h> #include <fcntl.h> #include <dirent.h> +#include <sys/mman.h> #include "util/debug.h" #include "util/parse-options.h"
@@ -21,6 +22,8 @@#include "util/thread_map.h" #include "util/cpumap.h" #include "util/trace-event.h" +#include "../lib/traceevent/kbuffer.h" +#include "../lib/traceevent/event-parse.h" #define DEFAULT_TRACER "function_graph"
@@ -31,6 +34,7 @@ struct perf_ftrace {struct perf_target target;const char *tracer;const char *dirname; +struct pevent *pevent; };static bool done;
@@ -786,6 +790,371 @@ out_reset: }static int +function_handler (struct trace_seq *s,struct pevent_record *record, +struct event_format *event,void *context__maybe_unused ) +{ +struct pevent *pevent = event->pevent; +unsigned long long function; +const char *func; + +if (pevent_get_field_val (s, event,"ip" , record, &function,1 )) +return trace_seq_putc (s,'!' ); + + func =pevent_find_function (pevent, function); +if (func) +trace_seq_printf (s,"%s <-- " , func); +else +trace_seq_printf (s,"0x%llx" , function); + +if (pevent_get_field_val (s, event,"parent_ip" , record, &function,1 )) +return trace_seq_putc (s,'!' ); + + func =pevent_find_function (pevent, function); +if (func) +trace_seq_printf (s,"%s" , func); +else +trace_seq_printf (s,"0x%llx" , function); + +trace_seq_putc (s,'\n' ); +return 0 ; +} + +#define TRACE_GRAPH_INDENT 2 + +static int +fgraph_ent_handler (struct trace_seq *s,struct pevent_record *record, +struct event_format *event,void *context__maybe_unused ) +{ +unsigned long long depth; +unsigned long long val; +const char *func; +int i; + +if (pevent_get_field_val (s, event,"depth" , record, &depth,1 )) +return trace_seq_putc (s,'!' ); + +/* Function */ +for (i =0 ; i < (int )(depth *TRACE_GRAPH_INDENT ); i++) +trace_seq_putc (s,' ' ); + +if (pevent_get_field_val (s, event,"func" , record, &val,1 )) +return trace_seq_putc (s,'!' ); + + func =pevent_find_function (event->pevent, val); + +if (func) +trace_seq_printf (s,"%s() {" , func); +else +trace_seq_printf (s,"%llx() {" , val); + +trace_seq_putc (s,'\n' ); +return 0 ; +} + +static int +fgraph_ret_handler (struct trace_seq *s,struct pevent_record *record, +struct event_format *event,void *context__maybe_unused ) +{ +unsigned long long depth; +int i; + +if (pevent_get_field_val (s, event,"depth" , record, &depth,1 )) +return trace_seq_putc (s,'!' ); + +/* Function */ +for (i =0 ; i < (int )(depth *TRACE_GRAPH_INDENT ); i++) +trace_seq_putc (s,' ' ); + +trace_seq_puts (s,"}\n" ); +return 0 ; +} + +struct perf_ftrace_report { +struct perf_ftrace *ftrace; +struct perf_tool tool; +}; + +struct ftrace_report_arg { +struct list_head node; +struct pevent_record *record; +struct kbuffer *kbuf; +void *map; +int cpu; +int fd; +off_t offset; +off_t size; +}; + +static LIST_HEAD (ftrace_cpu_buffers); + +static int process_sample_show (struct perf_tool *tool, +union perf_event * event__maybe_unused , +struct perf_sample *sample, +struct perf_evsel *evsel__maybe_unused , +struct machine *machine__maybe_unused ) +{ +struct perf_ftrace *ftrace; +struct perf_ftrace_report *show; +struct ftrace_report_arg *fra; +struct stat statbuf; +enum kbuffer_long_size long_size; +enum kbuffer_endian endian; +char buf[PATH_MAX ]; + + show = container_of(tool, struct perf_ftrace_report, tool); + ftrace = show->ftrace; + + fra =zalloc (sizeof (*fra)); +if (fra ==NULL ) +return -1 ; + + fra->cpu = sample->cpu; + +scnprintf (buf,sizeof (buf),"%s.dir/trace-cpu%d.buf" , + ftrace->dirname, fra->cpu); + + fra->fd =open (buf,O_RDONLY ); +if (fra->fd <0 ) + goto out; + +if (fstat (fra->fd, &statbuf) <0 ) + goto out_close; + + fra->size = statbuf.st_size; +if (fra->size ==0 ) { +/* skip zero-size buffers */ +free (fra); +return 0 ; + } + +/* +* FIXME: What if pevent->page_size is smaller than current page size? +*/ + fra->map =mmap (NULL ,pevent_get_page_size (ftrace->pevent), +PROT_READ ,MAP_PRIVATE , fra->fd, fra->offset); +if (fra->map ==MAP_FAILED ) + goto out_close; + + fra->offset =0 ; + +if (pevent_is_file_bigendian (ftrace->pevent)) + endian =KBUFFER_ENDIAN_BIG ; +else + endian =KBUFFER_ENDIAN_LITTLE ; + +if (pevent_get_long_size (ftrace->pevent) ==8 ) + long_size =KBUFFER_LSIZE_8 ; +else + long_size =KBUFFER_LSIZE_4 ; + + fra->kbuf =kbuffer_alloc (long_size, endian); +if (fra->kbuf ==NULL ) + goto out_unmap; + +if (ftrace->pevent->old_format) +kbuffer_set_old_format (fra->kbuf); + +kbuffer_load_subbuffer (fra->kbuf, fra->map); + +pr_debug2 ("setup kbuffer for cpu%d\n" , fra->cpu); +list_add_tail (&fra->node, &ftrace_cpu_buffers); +return 0 ; + +out_unmap: +munmap (fra->map,pevent_get_page_size (ftrace->pevent)); +out_close: +close (fra->fd); +out: +free (fra); +return -1 ; +} + +static struct pevent_record * +get_next_ftrace_event_record (struct perf_ftrace *ftrace, +struct ftrace_report_arg *fra) +{ +struct pevent_record *record; +unsigned long long ts; +void *data; + +retry: + data =kbuffer_read_event (fra->kbuf, &ts); +if (data) { + record =zalloc (sizeof (*record)); +if (record ==NULL ) { +pr_err ("memory allocation failure\n" ); +return NULL ; + } + + record->ts = ts; + record->cpu = fra->cpu; + record->data = data; + record->size =kbuffer_event_size (fra->kbuf); + record->record_size =kbuffer_curr_size (fra->kbuf); + record->offset =kbuffer_curr_offset (fra->kbuf); + record->missed_events =kbuffer_missed_events (fra->kbuf); + record->ref_count =1 ; + +kbuffer_next_event (fra->kbuf,NULL ); +return record; + } + +munmap (fra->map,pevent_get_page_size (ftrace->pevent)); + fra->map =NULL ; + + fra->offset +=pevent_get_page_size (ftrace->pevent); +if (fra->offset >= fra->size) { +/* EOF */ +return NULL ; + } + + fra->map =mmap (NULL ,pevent_get_page_size (ftrace->pevent), +PROT_READ ,MAP_PRIVATE , fra->fd, fra->offset); +if (fra->map ==MAP_FAILED ) { +pr_err ("memory mapping failed\n" ); +return NULL ; + } + +kbuffer_load_subbuffer (fra->kbuf, fra->map); + + goto retry; +} + +static struct pevent_record * +get_ftrace_event_record (struct perf_ftrace *ftrace, +struct ftrace_report_arg *fra) +{ +if (fra->record ==NULL ) + fra->record =get_next_ftrace_event_record (ftrace, fra); + +return fra->record; +} + +static struct pevent_record *get_ordered_record (struct perf_ftrace *ftrace) +{ +struct ftrace_report_arg *fra =NULL ; +struct ftrace_report_arg *tmp; +struct pevent_record *record; +unsigned long long min_ts =LLONG_MAX ; + +list_for_each_entry (tmp, &ftrace_cpu_buffers, node) { +if (perf_target__has_cpu (&ftrace->target)) { +int i; +bool found = false; + +for (i =0 ; i <cpu_map__nr (ftrace->evlist->cpus); i++) { +if (tmp->cpu == ftrace->evlist->cpus->map[i]) { + found = true; +break ; + } + } +if (!found) +continue ; + } + + record =get_ftrace_event_record (ftrace, tmp); +if (record && record->ts < min_ts) { + min_ts = record->ts; + fra = tmp; + } + } + +if (fra) { + record = fra->record; + fra->record =NULL ; +return record; + } +return NULL ; +} + +static void free_ftrace_report_args (struct perf_ftrace *ftrace) +{ +struct ftrace_report_arg *fra, *tmp; + +list_for_each_entry_safe (fra, tmp, &ftrace_cpu_buffers, node) { +list_del (&fra->node); + +/* don't care about the errors */ +munmap (fra->map,pevent_get_page_size (ftrace->pevent)); +kbuffer_free (fra->kbuf); +free (fra->record); +close (fra->fd); +free (fra); + } +} + +static int do_ftrace_show (struct perf_ftrace *ftrace) +{ +int ret =0 ; +char buf[PATH_MAX ]; +struct perf_session *session; +struct pevent_record *record; +struct trace_seq seq; +struct perf_ftrace_report show = { + .ftrace = ftrace, + .tool = { + .sample = process_sample_show, + }, + }; + +scnprintf (buf,sizeof (buf),"%s.dir/perf.header" , ftrace->dirname); + + session =perf_session__new (buf,O_RDONLY , false, false, &show.tool); +if (session ==NULL ) { +pr_err ("failed to create a session\n" ); +return -1 ; + } + + ftrace->pevent = session->pevent; + +pevent_register_event_handler (ftrace->pevent,-1 , +"ftrace" ,"function" , + function_handler,NULL ); +pevent_register_event_handler (ftrace->pevent,-1 , +"ftrace" ,"funcgraph_entry" , + fgraph_ent_handler,NULL ); +pevent_register_event_handler (ftrace->pevent,-1 , +"ftrace" ,"funcgraph_exit" , + fgraph_ret_handler,NULL ); + +if (perf_session__process_events (session, &show.tool) <0 ) { +pr_err ("failed to process events\n" ); + ret =-1 ; + goto out; + } + +trace_seq_init (&seq); + + record =get_ordered_record (ftrace); +while (record) { +int type; +struct event_format *event; + + type =pevent_data_type (ftrace->pevent, record); + event =pevent_find_event (ftrace->pevent, type); +if (!event) { +pr_warning ("no event found for type %d" , type); +continue ; + } + +pevent_print_event (ftrace->pevent, &seq, record); +trace_seq_do_printf (&seq); + +trace_seq_reset (&seq); + +free (record); + record =get_ordered_record (ftrace); + } + +trace_seq_destroy (&seq); + +out: +free_ftrace_report_args (ftrace); +perf_session__delete (session); +return ret; +} + +static int __cmd_ftrace_live (struct perf_ftrace *ftrace,int argc,const char **argv) {int ret =-1 ;
@@ -916,6 +1285,58 @@ out:return ret; } +static int +__cmd_ftrace_show (struct perf_ftrace *ftrace,int argc,const char **argv) +{ +int ret =-1 ; +const char *const show_usage[] = { +"perf ftrace show [<options>]" , +NULL + }; +const struct option show_options[] = { +OPT_STRING ('i' ,"input" , &ftrace->dirname,"dirname" , +"input directory name to use (default: perf.data)" ), +OPT_INCR ('v' ,"verbose" , &verbose, +"be more verbose" ), +OPT_STRING ('C' ,"cpu" , &ftrace->target.cpu_list,"cpu" , +"list of cpus to monitor" ), +OPT_END () + }; + + argc =parse_options (argc, argv, show_options, show_usage, +PARSE_OPT_STOP_AT_NON_OPTION ); +if (argc) +usage_with_options (show_usage, show_options); + + ret =perf_target__validate (&ftrace->target); +if (ret) { +char errbuf[512 ]; + +perf_target__strerror (&ftrace->target, ret, errbuf,512 ); +pr_err ("%s\n" , errbuf); +return -EINVAL ; + } + + ftrace->evlist =perf_evlist__new (); +if (ftrace->evlist ==NULL ) +return -ENOMEM ; + + ret =perf_evlist__create_maps (ftrace->evlist, &ftrace->target); +if (ret <0 ) + goto out; + +if (ftrace->dirname ==NULL ) + ftrace->dirname =DEFAULT_DIRNAME ; + + ret =do_ftrace_show (ftrace); + +perf_evlist__delete_maps (ftrace->evlist); +out: +perf_evlist__delete (ftrace->evlist); + +return ret; +} +int cmd_ftrace (int argc,const char **argv,const char *prefix__maybe_unused ) {int ret;
@@ -923,8 +1344,8 @@ int cmd_ftrace(int argc, const char **argv, const char *prefix __maybe_unused) .target = { .uid = UINT_MAX, }, };const char *const ftrace_usage[] = { -"perf ftrace {live|record} [<options>] [<command>]" , -"perf ftrace {live|record} [<options>] -- <command> [<options>]" , +"perf ftrace {live|record|show} [<options>] [<command>]" , +"perf ftrace {live|record|show} [<options>] -- <command> [<options>]" ,NULL };const struct option ftrace_options[] = {
@@ -945,6 +1366,8 @@ int cmd_ftrace(int argc, const char **argv, const char *prefix __maybe_unused) ret =__cmd_ftrace_live (&ftrace, argc, argv); }else if (strncmp (argv[0 ],"rec" ,3 ) ==0 ) { ret =__cmd_ftrace_record (&ftrace, argc, argv); + }else if (strcmp (argv[0 ],"show" ) ==0 ) { + ret =__cmd_ftrace_show (&ftrace, argc, argv); }else {usage_with_options (ftrace_usage, ftrace_options); }
Signature · 1 line
1.7.11.7
Re: [PATCH 10/15] perf ftrace: Add 'show' sub-command
Quoted 2 lines · Namhyung Kim
From: Namhyung Kim <[email protected]>
SNIP
Quoted 9 lines
tools/perf/builtin-ftrace.c +9
-0+ +static LIST_HEAD (ftrace_cpu_buffers); + +static int process_sample_show (struct perf_tool *tool, +union perf_event * event__maybe_unused , +struct perf_sample *sample, +struct perf_evsel *evsel__maybe_unused , +struct machine *machine__maybe_unused ) +{
tools/perf/builtin-ftrace.c out_reset: · line 905
Quoted 10 lines
tools/perf/builtin-ftrace.c +10
-0+struct perf_ftrace *ftrace; +struct perf_ftrace_report *show; +struct ftrace_report_arg *fra; +struct stat statbuf; +enum kbuffer_long_size long_size; +enum kbuffer_endian endian; +char buf[PATH_MAX ]; + + show = container_of(tool, struct perf_ftrace_report, tool); + ftrace = show->ftrace;
this seems to be exactly same routine as process_sample_report with exception of above 2 lines.. maybe the rest of it could be common function?
jirka
Namhyung Kim
maintainerRe: [PATCH 10/15] perf ftrace: Add 'show' sub-command
Quoted 13 lines · Jiri Olsa
Quoted 12 lines · Namhyung Kim
tools/perf/builtin-ftrace.c +12
-0+static int process_sample_show (struct perf_tool *tool, +union perf_event * event__maybe_unused , +struct perf_sample *sample, +struct perf_evsel *evsel__maybe_unused , +struct machine *machine__maybe_unused ) +{ +struct perf_ftrace *ftrace; +struct perf_ftrace_report *show; +struct ftrace_report_arg *fra; +struct stat statbuf; +enum kbuffer_long_size long_size; +enum kbuffer_endian endian;
Quoted 8 lines
tools/perf/builtin-ftrace.c out_reset: · line 905
Quoted 4 lines
tools/perf/builtin-ftrace.c +4
-0+char buf[PATH_MAX ]; + + show = container_of(tool, struct perf_ftrace_report, tool); + ftrace = show->ftrace;this seems to be exactly same routine as
process_sample_reportwith exception of above 2 lines.. maybe the rest of it could be common function?
Hmm.. indeed. I'd rather use just one of them as only thing they do is to get the ftrace pointer.
Thanks for pointing this out,
Namhyung
Namhyung Kim
maintainerPATCH 11/15 perf ftrace: Add 'report' sub-command
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
The ftrace report command is for analyzing ftrace result as usual perf report style. Internal processing of the ftrace buffer is similar to the 'show' sub-command but it synthesizes necessary information like thread, dso, map and symbol from saved trace info.
It currently count number of samples as a period and it can be extended to use calltime of funcgraph_exit in the future.
--- tools/perf/builtin-ftrace.c | 363 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 361 insertions(+), 2 deletions(-)
tools/perf/builtin-ftrace.c +361 -2
@@ -21,6 +21,7 @@#include "util/target.h" #include "util/thread_map.h" #include "util/cpumap.h" +#include "util/sort.h" #include "util/trace-event.h" #include "../lib/traceevent/kbuffer.h" #include "../lib/traceevent/event-parse.h"
@@ -35,6 +36,7 @@ struct perf_ftrace {const char *tracer;const char *dirname;struct pevent *pevent; +bool show_full_info; };static bool done;
@@ -1154,6 +1156,294 @@ out:return ret; } +static int process_sample_report (struct perf_tool *tool, +union perf_event * event__maybe_unused , +struct perf_sample *sample, +struct perf_evsel *evsel__maybe_unused , +struct machine *machine__maybe_unused ) +{ +struct perf_ftrace *ftrace; +struct perf_ftrace_report *report; +struct ftrace_report_arg *fra; +struct stat statbuf; +enum kbuffer_long_size long_size; +enum kbuffer_endian endian; +char buf[PATH_MAX ]; + + report = container_of(tool, struct perf_ftrace_report, tool); + ftrace = report->ftrace; + + fra =zalloc (sizeof (*fra)); +if (fra ==NULL ) +return -1 ; + + fra->cpu = sample->cpu; + +scnprintf (buf,sizeof (buf),"%s.dir/trace-cpu%d.buf" , + ftrace->dirname, fra->cpu); + + fra->fd =open (buf,O_RDONLY ); +if (fra->fd <0 ) + goto out; + +if (fstat (fra->fd, &statbuf) <0 ) + goto out_close; + + fra->size = statbuf.st_size; +if (fra->size ==0 ) { +/* skip zero-size buffers */ +free (fra); +return 0 ; + } + +/* +* FIXME: What if pevent->page_size is smaller than current page size? +*/ + fra->map =mmap (NULL ,pevent_get_page_size (ftrace->pevent), +PROT_READ ,MAP_PRIVATE , fra->fd, fra->offset); +if (fra->map ==MAP_FAILED ) + goto out_close; + + fra->offset =0 ; + +if (pevent_is_file_bigendian (ftrace->pevent)) + endian =KBUFFER_ENDIAN_BIG ; +else + endian =KBUFFER_ENDIAN_LITTLE ; + +if (pevent_get_long_size (ftrace->pevent) ==8 ) + long_size =KBUFFER_LSIZE_8 ; +else + long_size =KBUFFER_LSIZE_4 ; + + fra->kbuf =kbuffer_alloc (long_size, endian); +if (fra->kbuf ==NULL ) + goto out_unmap; + +if (ftrace->pevent->old_format) +kbuffer_set_old_format (fra->kbuf); + +kbuffer_load_subbuffer (fra->kbuf, fra->map); + +pr_debug2 ("setup kbuffer for cpu%d\n" , fra->cpu); +list_add_tail (&fra->node, &ftrace_cpu_buffers); +return 0 ; + +out_unmap: +munmap (fra->map,pevent_get_page_size (ftrace->pevent)); +out_close: +close (fra->fd); +out: +free (fra); +return -1 ; +} + +struct cmdline_list { +struct cmdline_list *next; +char *comm; +int pid; +}; + +struct func_list { +struct func_list *next; +unsigned long long addr; +char *func; +char *mod; +}; + +static int do_ftrace_report (struct perf_ftrace *ftrace) +{ +int ret =-1 ; +char buf[PATH_MAX ]; +unsigned long nr_samples; +struct perf_session *session; +struct perf_evsel *evsel; +struct pevent_record *record; +struct perf_ftrace_report report = { + .ftrace = ftrace, + .tool = { + .sample = process_sample_report, + }, + }; +struct cmdline_list *cmdline; +struct func_list *func; +struct machine *machine; +struct dso *dso; + +scnprintf (buf,sizeof (buf),"%s.dir/perf.header" , ftrace->dirname); + + session =perf_session__new (buf,O_RDONLY , false, false, &report.tool); +if (session ==NULL ) { +pr_err ("failed to create a session\n" ); +return -1 ; + } + + ftrace->pevent = session->pevent; + +if (perf_session__process_events (session, &report.tool) <0 ) { +pr_err ("failed to process events\n" ); + goto out; + } + + machine =machines__findnew (&session->machines,HOST_KERNEL_ID ); + +/* Synthesize thread info from saved cmdlines */ + cmdline = ftrace->pevent->cmdlist; +while (cmdline) { +struct thread *thread; + + thread =machine__findnew_thread (machine, cmdline->pid); +if (thread && !thread->comm_set) +thread__set_comm (thread, cmdline->comm); + + cmdline = cmdline->next; + } + +/* Synthesize kernel dso and symbol info from saved kallsyms */ + func = ftrace->pevent->funclist; +while (func) { +struct symbol *sym; + +scnprintf (buf,sizeof (buf),"[%s]" , + func->mod ? func->mod :"kernel.kallsyms" ); + + dso =dso__kernel_findnew (machine, buf,NULL ,DSO_TYPE_KERNEL ); +if (dso ==NULL ) { +pr_debug ("can't find or allocate dso %s\n" , buf); +continue ; + } + + sym =symbol__new (func->addr,0 ,STB_GLOBAL , func->func); +if (sym ==NULL ) { +pr_debug ("failed to allocate new symbol\n" ); +continue ; + } +symbols__insert (&dso->symbols[MAP__FUNCTION ], sym); + + func = func->next; + } + +/* Generate kernel maps */ +list_for_each_entry (dso, &machine->kernel_dsos, node) { +struct map *map =map__new2 (0 , dso,MAP__FUNCTION ); +if (map ==NULL ) { +pr_debug ("failed to allocate new map\n" ); + goto out; + } + +symbols__fixup_end (&dso->symbols[MAP__FUNCTION ]); +map__fixup_start (map); +map__fixup_end (map); + +dso__set_loaded (dso,MAP__FUNCTION ); + +map_groups__insert (&machine->kmaps, map); +if (strcmp (dso->name,"[kernel.kallsyms]" ) ==0 ) + machine->vmlinux_maps[MAP__FUNCTION ] = map; + } + +/* FIXME: no need to get ordered */ + record =get_ordered_record (ftrace); +while (record) { +int type; +struct addr_location al; +union perf_event event = { + .header = { + .misc =PERF_RECORD_MISC_KERNEL , + }, + }; +struct perf_sample sample = { + .cpu = record->cpu, + .raw_data = record->data, + .period =1 , + }; +struct format_field *field; +unsigned long long val; + + type =pevent_data_type (ftrace->pevent, record); + evsel =perf_evlist__find_tracepoint_by_id (session->evlist, + type); +if (evsel ==NULL ) { +pr_warning ("no event found for type %d\n" , type); +continue ; + } + + event.ip.pid =pevent_data_pid (ftrace->pevent, record); + +if (!strcmp (perf_evsel__name (evsel),"ftrace:function" )) + field =pevent_find_field (evsel->tp_format,"ip" ); +else + field =pevent_find_field (evsel->tp_format,"func" ); + +if (pevent_read_number_field (field, record->data, &val) <0 ) { +pr_err ("failed to parse function address\n" ); + goto out; + } + event.ip.ip = val; + +if (perf_event__preprocess_sample (&event, machine, &al, + &sample,NULL ) <0 ) { +pr_err ("problem processing %d event, skipping it.\n" , + event.header.type); + goto out; + } + +/* TODO: update sample.period using calltime */ +if (!__hists__add_entry (&evsel->hists, &al,NULL , + sample.period,0 )) { +pr_err ("failed to add a hist entry\n" ); + goto out; + } + + evsel->hists.stats.total_period += sample.period; +hists__inc_nr_events (&evsel->hists,PERF_RECORD_SAMPLE ); + +free (record); + record =get_ordered_record (ftrace); + } + ret =0 ; + +perf_session__fprintf_info (session, stdout, ftrace->show_full_info); + + nr_samples =0 ; +list_for_each_entry (evsel, &session->evlist->entries, node) { +struct hists *hists = &evsel->hists; + +hists__collapse_resort (hists); +hists__output_resort (&evsel->hists); + nr_samples += hists->stats.nr_events[PERF_RECORD_SAMPLE ]; + } + +if (nr_samples ==0 ) { +pr_warning ("The %s file has no samples!\n" , session->filename); + goto out; + } + +list_for_each_entry (evsel, &session->evlist->entries, node) { +struct hists *hists = &evsel->hists; +const char *evname =perf_evsel__name (evsel); +u64 nr_events = hists->stats.total_period; +char unit; + + nr_samples = hists->stats.nr_events[PERF_RECORD_SAMPLE ]; + nr_samples =convert_unit (nr_samples, &unit); +fprintf (stdout,"# Samples: %lu%c" , nr_samples, unit); +if (evname !=NULL ) +fprintf (stdout," of event '%s'" , evname); + +fprintf (stdout,"\n# Event count (approx.): %" PRIu64, nr_events); +fprintf (stdout,"\n#\n" ); + +hists__fprintf (hists, true,0 ,0 , stdout); +fprintf (stdout,"\n\n" ); + } + +out: +free_ftrace_report_args (ftrace); +perf_session__delete (session); +return ret; +} +static int __cmd_ftrace_live (struct perf_ftrace *ftrace,int argc,const char **argv) {
@@ -1337,6 +1627,73 @@ out:return ret; } +static int +__cmd_ftrace_report (struct perf_ftrace *ftrace,int argc,const char **argv) +{ +int ret =-1 ; +const char *const report_usage[] = { +"perf ftrace report [<options>]" , +NULL + }; +const struct option report_options[] = { +OPT_STRING ('i' ,"input" , &ftrace->dirname,"dirname" , +"input directory name to use (default: perf.data)" ), +OPT_INCR ('v' ,"verbose" , &verbose, +"be more verbose" ), +OPT_BOOLEAN ('D' ,"dump-raw-trace" , &dump_trace, +"dump raw trace in ASCII" ), +OPT_STRING ('C' ,"cpu" , &ftrace->target.cpu_list,"cpu" , +"list of cpus to monitor" ), +OPT_STRING ('s' ,"sort" , &sort_order,"key[,key2...]" , +"sort by key(s): pid, comm, dso, symbol, cpu" ), +OPT_BOOLEAN ('I' ,"show-info" , &ftrace->show_full_info, +"Display extended information like cpu/numa topology" ), +OPT_END () + }; + + argc =parse_options (argc, argv, report_options, report_usage, +PARSE_OPT_STOP_AT_NON_OPTION ); +if (argc) +usage_with_options (report_usage, report_options); + + ret =perf_target__validate (&ftrace->target); +if (ret) { +char errbuf[512 ]; + +perf_target__strerror (&ftrace->target, ret, errbuf,512 ); +pr_err ("%s\n" , errbuf); +return -EINVAL ; + } + + ftrace->evlist =perf_evlist__new (); +if (ftrace->evlist ==NULL ) +return -ENOMEM ; + + ret =perf_evlist__create_maps (ftrace->evlist, &ftrace->target); +if (ret <0 ) + goto out; + +if (ftrace->dirname ==NULL ) + ftrace->dirname =DEFAULT_DIRNAME ; + +perf_hpp__column_enable (PERF_HPP__OVERHEAD ); +perf_hpp__init (); + +setup_sorting (); + + symbol_conf.exclude_other = false; + symbol_conf.try_vmlinux_path = false; +symbol__init (); + + ret =do_ftrace_report (ftrace); + +perf_evlist__delete_maps (ftrace->evlist); +out: +perf_evlist__delete (ftrace->evlist); + +return ret; +} +int cmd_ftrace (int argc,const char **argv,const char *prefix__maybe_unused ) {int ret;
@@ -1344,8 +1701,8 @@ int cmd_ftrace(int argc, const char **argv, const char *prefix __maybe_unused) .target = { .uid = UINT_MAX, }, };const char *const ftrace_usage[] = { -"perf ftrace {live|record|show} [<options>] [<command>]" , -"perf ftrace {live|record|show} [<options>] -- <command> [<options>]" , +"perf ftrace {live|record|show|report} [<options>] [<command>]" , +"perf ftrace {live|record|show|report} [<options>] -- <command> [<options>]" ,NULL };const struct option ftrace_options[] = {
@@ -1368,6 +1725,8 @@ int cmd_ftrace(int argc, const char **argv, const char *prefix __maybe_unused) ret =__cmd_ftrace_record (&ftrace, argc, argv); }else if (strcmp (argv[0 ],"show" ) ==0 ) { ret =__cmd_ftrace_show (&ftrace, argc, argv); + }else if (strncmp (argv[0 ],"rep" ,3 ) ==0 ) { + ret =__cmd_ftrace_report (&ftrace, argc, argv); }else {usage_with_options (ftrace_usage, ftrace_options); }
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 12/15 perf ftrace: Use pager for displaying result
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
It's convenient to use pager when seeing many lines of result.
--- tools/perf/builtin-ftrace.c | 6 ++++++ 1 file changed, 6 insertions(+)
tools/perf/builtin-ftrace.c +6 -0
@@ -227,6 +227,7 @@ static int do_ftrace_live(struct perf_ftrace *ftrace)signal (SIGINT , sig_handler);signal (SIGUSR1 , sig_handler);signal (SIGCHLD , sig_handler); +signal (SIGPIPE , sig_handler);if (setup_tracing_files (ftrace) <0 ) goto out_reset;
@@ -1472,6 +1473,8 @@ __cmd_ftrace_live(struct perf_ftrace *ftrace, int argc, const char **argv)if (!argc &&perf_target__none (&ftrace->target))usage_with_options (live_usage, live_options); +setup_pager (); + ret =perf_target__validate (&ftrace->target);if (ret) {char errbuf[512 ];
@@ -1618,6 +1621,8 @@ __cmd_ftrace_show(struct perf_ftrace *ftrace, int argc, const char **argv)if (ftrace->dirname ==NULL ) ftrace->dirname =DEFAULT_DIRNAME ; +setup_pager (); + ret =do_ftrace_show (ftrace);perf_evlist__delete_maps (ftrace->evlist);
@@ -1679,6 +1684,7 @@ __cmd_ftrace_report(struct perf_ftrace *ftrace, int argc, const char **argv)perf_hpp__column_enable (PERF_HPP__OVERHEAD );perf_hpp__init (); +setup_pager ();setup_sorting (); symbol_conf.exclude_other = false;
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 13/15 perf ftrace: Cleanup using ftrace_setup/teardown()
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
The ftrace sub-commands share some common code so that factor it out to ftrace_setup() and ftrace_teardown() helpers.
--- tools/perf/builtin-ftrace.c | 174 +++++++++++++++++--------------------------- 1 file changed, 67 insertions(+), 107 deletions(-)
tools/perf/builtin-ftrace.c +67 -107
@@ -1445,10 +1445,59 @@ out:return ret; } +static int ftrace_setup (struct perf_ftrace *ftrace,int argc,const char **argv) +{ +int ret; +char errbuf[512 ]; + + ret =perf_target__validate (&ftrace->target); +if (ret) { +perf_target__strerror (&ftrace->target, ret, errbuf,512 ); +pr_err ("%s\n" , errbuf); +return -EINVAL ; + } + + ftrace->evlist =perf_evlist__new (); +if (ftrace->evlist ==NULL ) +return -ENOMEM ; + + ret =perf_evlist__create_maps (ftrace->evlist, &ftrace->target); +if (ret <0 ) + goto out; + +if (ftrace->tracer ==NULL ) + ftrace->tracer =DEFAULT_TRACER ; + +if (ftrace->dirname ==NULL ) + ftrace->dirname =DEFAULT_DIRNAME ; + +if (argc) { + ret =perf_evlist__prepare_workload (ftrace->evlist, + &ftrace->target, + argv, false, true); +if (ret <0 ) + goto out_maps; + } +return 0 ; + +out_maps: +perf_evlist__delete_maps (ftrace->evlist); +out: +perf_evlist__delete (ftrace->evlist); + +return ret; +} + +static void ftrace_teardown (struct perf_ftrace *ftrace) +{ +perf_evlist__delete_maps (ftrace->evlist); +perf_evlist__delete (ftrace->evlist); +} +static int __cmd_ftrace_live (struct perf_ftrace *ftrace,int argc,const char **argv) { -int ret =-1 ; +int ret;const char *const live_usage[] = {"perf ftrace live [<options>] [<command>]" ,"perf ftrace live [<options>] -- <command> [<options>]" ,
@@ -1475,45 +1524,20 @@ __cmd_ftrace_live(struct perf_ftrace *ftrace, int argc, const char **argv)setup_pager (); - ret =perf_target__validate (&ftrace->target); -if (ret) { -char errbuf[512 ]; - -perf_target__strerror (&ftrace->target, ret, errbuf,512 ); -pr_err ("%s\n" , errbuf); -return -EINVAL ; - } - - ftrace->evlist =perf_evlist__new (); -if (ftrace->evlist ==NULL ) -return -ENOMEM ; - - ret =perf_evlist__create_maps (ftrace->evlist, &ftrace->target); + ret =ftrace_setup (ftrace, argc, argv);if (ret <0 ) - goto out; - -if (ftrace->tracer ==NULL ) - ftrace->tracer =DEFAULT_TRACER ; - -if (argc &&perf_evlist__prepare_workload (ftrace->evlist, - &ftrace->target, - argv, false, true) <0 ) - goto out_maps; +return ret; ret =do_ftrace_live (ftrace); -out_maps: -perf_evlist__delete_maps (ftrace->evlist); -out: -perf_evlist__delete (ftrace->evlist); - +ftrace_teardown (ftrace);return ret; }static int __cmd_ftrace_record (struct perf_ftrace *ftrace,int argc,const char **argv) { -int ret =-1 ; +int ret;const char *const record_usage[] = {"perf ftrace record [<options>] [<command>]" ,"perf ftrace record [<options>] -- <command> [<options>]" ,
@@ -1540,39 +1564,13 @@ __cmd_ftrace_record(struct perf_ftrace *ftrace, int argc, const char **argv)if (!argc &&perf_target__none (&ftrace->target))usage_with_options (record_usage, record_options); - ret =perf_target__validate (&ftrace->target); -if (ret) { -char errbuf[512 ]; - -perf_target__strerror (&ftrace->target, ret, errbuf,512 ); -pr_err ("%s\n" , errbuf); -return -EINVAL ; - } - - ftrace->evlist =perf_evlist__new (); -if (ftrace->evlist ==NULL ) -return -ENOMEM ; - - ret =perf_evlist__create_maps (ftrace->evlist, &ftrace->target); + ret =ftrace_setup (ftrace, argc, argv);if (ret <0 ) - goto out; - -if (ftrace->tracer ==NULL ) - ftrace->tracer =DEFAULT_TRACER ; - -if (ftrace->dirname ==NULL ) - ftrace->dirname =DEFAULT_DIRNAME ; - -if (argc &&perf_evlist__prepare_workload (ftrace->evlist, - &ftrace->target, - argv, false, true) <0 ) - goto out_maps; +return ret; ret =do_ftrace_record (ftrace); -out_maps:perf_evlist__delete_maps (ftrace->evlist); -out:perf_evlist__delete (ftrace->evlist);return ret;
@@ -1581,7 +1579,7 @@ out:static int __cmd_ftrace_show (struct perf_ftrace *ftrace,int argc,const char **argv) { -int ret =-1 ; +int ret;const char *const show_usage[] = {"perf ftrace show [<options>]" ,NULL
@@ -1601,41 +1599,22 @@ __cmd_ftrace_show(struct perf_ftrace *ftrace, int argc, const char **argv)if (argc)usage_with_options (show_usage, show_options); - ret =perf_target__validate (&ftrace->target); -if (ret) { -char errbuf[512 ]; - -perf_target__strerror (&ftrace->target, ret, errbuf,512 ); -pr_err ("%s\n" , errbuf); -return -EINVAL ; - } - - ftrace->evlist =perf_evlist__new (); -if (ftrace->evlist ==NULL ) -return -ENOMEM ; +setup_pager (); - ret =perf_evlist__create_maps (ftrace->evlist, &ftrace->target); + ret =ftrace_setup (ftrace, argc, argv);if (ret <0 ) - goto out; - -if (ftrace->dirname ==NULL ) - ftrace->dirname =DEFAULT_DIRNAME ; - -setup_pager (); +return ret; ret =do_ftrace_show (ftrace); -perf_evlist__delete_maps (ftrace->evlist); -out: -perf_evlist__delete (ftrace->evlist); - +ftrace_teardown (ftrace);return ret; }static int __cmd_ftrace_report (struct perf_ftrace *ftrace,int argc,const char **argv) { -int ret =-1 ; +int ret;const char *const report_usage[] = {"perf ftrace report [<options>]" ,NULL
@@ -1661,26 +1640,6 @@ __cmd_ftrace_report(struct perf_ftrace *ftrace, int argc, const char **argv)if (argc)usage_with_options (report_usage, report_options); - ret =perf_target__validate (&ftrace->target); -if (ret) { -char errbuf[512 ]; - -perf_target__strerror (&ftrace->target, ret, errbuf,512 ); -pr_err ("%s\n" , errbuf); -return -EINVAL ; - } - - ftrace->evlist =perf_evlist__new (); -if (ftrace->evlist ==NULL ) -return -ENOMEM ; - - ret =perf_evlist__create_maps (ftrace->evlist, &ftrace->target); -if (ret <0 ) - goto out; - -if (ftrace->dirname ==NULL ) - ftrace->dirname =DEFAULT_DIRNAME ; -perf_hpp__column_enable (PERF_HPP__OVERHEAD );perf_hpp__init ();
@@ -1691,12 +1650,13 @@ __cmd_ftrace_report(struct perf_ftrace *ftrace, int argc, const char **argv) symbol_conf.try_vmlinux_path = false;symbol__init (); - ret =do_ftrace_report (ftrace); + ret =ftrace_setup (ftrace, argc, argv); +if (ret <0 ) +return ret; -perf_evlist__delete_maps (ftrace->evlist); -out: -perf_evlist__delete (ftrace->evlist); + ret =do_ftrace_report (ftrace); +ftrace_teardown (ftrace);return ret; }
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 14/15 perf tools: Add document for perf-ftrace command
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
--- tools/perf/Documentation/perf-ftrace.txt | 114 +++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100644 tools/perf/Documentation/perf-ftrace.txt
tools/perf/Documentation/perf-ftrace.txt new file +114 -0
@@ -0,0 +1,114 @@ +perf-ftrace(1) +============== + +NAME +---- +perf-ftrace - A front end for kernel's ftrace + +SYNOPSIS +-------- +[verse] +'perf ftrace' {live|record} [<options>] [<command>] +'perf ftrace' {show|report} [<options>] + +DESCRIPTION +----------- +This command reads the ftrace buffer and displays the trace recorded. + +There are several variants of perf ftrace: + + 'perf ftrace live <command>' to see a live trace of kernel functions + via trace_pipe during executing the <command>. If <command> is not + specified, one of the target options (-p, -a or -C) should be given. + It just prints out the result to stdout and doesn't save any files. + + 'perf ftrace record <command>' to record trace entries of kernel + functions during the execution of the <command>. Like 'perf ftrace + live', at least one of <command> and/or target options should be + given in order to start recording. The recorded results are saved + under a directory ('perf.data.dir' by default) and will be used by + other perf-ftrace tools like 'show' and 'report'. + + 'perf ftrace show' to see the trace of recorded functions. It shows + functions sorted by time so the end result might be interspersed if + there's a concurrent execution. + + 'perf ftrace report' to display the result in an usual perf-report + style - entries are sorted by given sort keys and output is resorted + by its overhead. + +OPTIONS +------- +<command>...:: + Any command you can specify in a shell. + +-t:: +--tracer=:: + The ftrace tracer to be used (default: function_graph). + Currently only 'function' and 'function_graph' are supported. + Used by 'live' and 'record' subcommands only. + +-p:: +--pid=:: + Record events on existing process ID (comma separated list). + Used by 'live' and 'record' subcommands only. + +-a:: + Force system-wide collection. Scripts run without a <command> + normally use -a by default, while scripts run with a <command> + normally don't - this option allows the latter to be run in + system-wide mode. Used by 'live' and 'record' subcommands only. + +-C:: +--cpu:: Only process samples for the list of CPUs provided. + Multiple CPUs can be provided as a comma-separated list with + no space: 0,1. Ranges of CPUs are specified with -: 0-2. + Default is to report samples on all online CPUs. + +-s:: +--sort=:: + Sort histogram entries by given key(s) - multiple keys can be + specified in CSV format. Following sort keys are available: + pid, comm, dso, symbol, cpu. + + Each key has following meaning: + + - comm: command (name) of the task + - pid: command and tid of the task + - dso: name of library or module executed at the time of sample + - symbol: name of function executed at the time of sample + - cpu: cpu number the task ran at the time of sample + + By default, comm, dso and symbol keys are used. + (i.e. --sort comm,dso,symbol) + Used by 'report' subcommands only. + +-i:: +--input=:: + Input directory name excluding '.dir' at the end. + (default: perf.data) + +-o:: +--output=:: + Output directory name excluding '.dir' at the end. + (default: perf.data) + +-v:: +--verbose:: + Be more verbose (show counter open errors, etc). + +-D:: +--dump-raw-trace:: + Dump raw trace in ASCII. Used by 'report' subcommands only. + +-I:: +--show-info:: + Display extended information about the record. This adds + information which may be very large and thus may clutter the + display. It currently includes: cpu and numa topology of the + host system. It can only be used with the 'report' subcommand. + +SEE ALSO +-------- +linkperf:perf-record[1], linkperf:perf-report[1], +linkperf:perf-script[1]
Signature · 1 line
1.7.11.7
Namhyung Kim
maintainerPATCH 15/15 perf ftrace: Add a signal handler for SIGSEGV
In reply to Namhyung KimFrom: Namhyung Kim <[email protected]>
It's for debugging purpose.
--- tools/perf/builtin-ftrace.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
tools/perf/builtin-ftrace.c +13 -0
@@ -41,6 +41,13 @@ struct perf_ftrace {static bool done; +static void sig_exit (int sig) +{ +psignal (sig,"perf" ); +signal (sig,SIG_DFL ); +raise (sig); +} +static void sig_handler (int sig__maybe_unused ) { done = true;
@@ -228,6 +235,7 @@ static int do_ftrace_live(struct perf_ftrace *ftrace)signal (SIGUSR1 , sig_handler);signal (SIGCHLD , sig_handler);signal (SIGPIPE , sig_handler); +signal (SIGSEGV , sig_exit);if (setup_tracing_files (ftrace) <0 ) goto out_reset;
@@ -602,6 +610,7 @@ static int do_ftrace_record(struct perf_ftrace *ftrace)signal (SIGINT , sig_handler);signal (SIGUSR1 , sig_handler);signal (SIGCHLD , sig_handler); +signal (SIGSEGV , sig_exit);if (setup_tracing_files (ftrace) <0 ) goto out_reset;
@@ -1100,6 +1109,8 @@ static int do_ftrace_show(struct perf_ftrace *ftrace) }, }; +signal (SIGSEGV , sig_exit); +scnprintf (buf,sizeof (buf),"%s.dir/perf.header" , ftrace->dirname); session =perf_session__new (buf,O_RDONLY , false, false, &show.tool);
@@ -1271,6 +1282,8 @@ static int do_ftrace_report(struct perf_ftrace *ftrace)struct machine *machine;struct dso *dso; +signal (SIGSEGV , sig_exit); +scnprintf (buf,sizeof (buf),"%s.dir/perf.header" , ftrace->dirname); session =perf_session__new (buf,O_RDONLY , false, false, &report.tool);
Signature · 1 line
1.7.11.7