perf tools: Support dynamic sort keys for tracepoints (v1)
Namhyung Kim
maintainercover perf tools: Support dynamic sort keys for tracepoints (v1)
Hello,
This is an attempt to improve perf to deal with tracepoint events better. The perf tools can handle tracepoint events but perf report on them is less useful since they're always sampled in a fixed location and not provide event specific info. We can use perf script but I always wishes there's more convenient way to see the result.
I suggest dynamic sort keys created for each event.field on demand. Consider following example:
# Overhead Command Shared Object Symbol
# ........ ............... ................ ..............
#
47.22% swapper [kernel.vmlinux] [k] __schedule
21.67% transmission-gt [kernel.vmlinux] [k] __schedule
8.23% netctl-auto [kernel.vmlinux] [k] __schedule
5.53% kworker/0:1H [kernel.vmlinux] [k] __schedule
1.98% Xephyr [kernel.vmlinux] [k] __schedule
1.33% irq/33-iwlwifi [kernel.vmlinux] [k] __schedule
1.17% wpa_cli [kernel.vmlinux] [k] __schedule
1.13% rcu_preempt [kernel.vmlinux] [k] __schedule
0.85% ksoftirqd/0 [kernel.vmlinux] [k] __schedule
0.77% Timer [kernel.vmlinux] [k] __schedule...
Currently perf report only shows this but important info is on the event fields, that is:
# sudo cat /sys/kernel/debug/tracing/events/sched/sched_switch/format
name: sched_switch
ID: 268
format:
field:unsigned short common_type; offset:0; size:2; signed:0;
field:unsigned char common_flags; offset:2; size:1; signed:0;
field:unsigned char common_preempt_count; offset:3; size:1; signed:0;
field:int common_pid; offset:4; size:4; signed:1; field:char prev_comm[16]; offset:8; size:16; signed:1;
field:pid_t prev_pid; offset:24; size:4; signed:1;
field:int prev_prio; offset:28; size:4; signed:1;
field:long prev_state; offset:32; size:8; signed:1;
field:char next_comm[16]; offset:40; size:16; signed:1;
field:pid_t next_pid; offset:56; size:4; signed:1;
field:int next_prio; offset:60; size:4; signed:1;print fmt: "prev_comm=%s prev_pid=%d prev_prio=%d prev_state=%s%s ==>
next_comm=%s next_pid=%d next_prio=%d",
REC->prev_comm, REC->prev_pid, REC->prev_prio,
REC->prev_state & (2048-1) ? __print_flags(REC->prev_state & (2048-1),
"|", { 1, "S"} , { 2, "D" }, { 4, "T" }, { 8, "t" }, { 16, "Z" }, { 32, "X" },
{ 64, "x" }, { 128, "K"}, { 256, "W" }, { 512, "P" }, { 1024, "N" }) : "R",
REC->prev_state & 2048 ? "+" : "", REC->next_comm, REC->next_pid, REC->next_prioWith dynamic sort keys, you can use <event.field> as a sort key. Those dynamic keys are checked and created on demand. For instance, below is to sort by next_pid field on the same data file.
$ perf report -s comm,sched:sched_switch.next_pid --stdio
...
# Overhead Command next_pid
# ........ ............... ..........
#
21.23% transmission-gt 0
20.86% swapper 17773
6.62% netctl-auto 0
5.25% swapper 109
5.21% kworker/0:1H 0
1.98% Xephyr 0
1.98% swapper 6524
1.98% swapper 27478
1.37% swapper 27476
1.17% swapper 233Multiple dynamic sort keys are also supported:
$ perf report -s comm,sched:sched_switch.next_pid,sched:sched_switch.next_comm --stdio
...
# Overhead Command next_pid next_comm
# ........ ............... .......... ................
#
20.86% swapper 17773 transmission-gt
9.64% transmission-gt 0 swapper/0
9.16% transmission-gt 0 swapper/2
5.25% swapper 109 kworker/0:1H
5.21% kworker/0:1H 0 swapper/0
2.14% netctl-auto 0 swapper/2
1.98% netctl-auto 0 swapper/0
1.98% swapper 6524 Xephyr
1.98% swapper 27478 netctl-auto
1.78% transmission-gt 0 swapper/3
1.53% Xephyr 0 swapper/0
1.29% netctl-auto 0 swapper/1
1.29% swapper 27476 netctl-auto
1.21% netctl-auto 0 swapper/3
1.17% swapper 233 irq/33-iwlwifiNote that pid 0 exists for each cpu so have comm of 'swapper/N'.
This is available on 'perf/dynamic-sort-v1' branch in my tree
git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
Any comments are welcome, thanks!
Namhyung
Namhyung Kim (6):
perf hist: Pass struct sample to __hists__add_entry() perf hist: Save raw_data/size for tracepoint events tools lib traceevent: Factor out and export print_event_field() perf tools: Pass evlist to setup_sorting()
perf tools: Add dynamic sort key for tracepoint events perf tools: Try to show pretty printed output for dynamic sort keys
tools/lib/traceevent/event-parse.c | 124 ++++++++------- tools/lib/traceevent/event-parse.h | 2 + tools/perf/builtin-annotate.c | 9 +- tools/perf/builtin-diff.c | 13 +- tools/perf/builtin-report.c | 2 +- tools/perf/builtin-top.c | 14 +- tools/perf/tests/hists_cumulate.c | 8 +- tools/perf/tests/hists_filter.c | 2 +- tools/perf/tests/hists_link.c | 8 +- tools/perf/tests/hists_output.c | 10 +- tools/perf/util/hist.c | 33 ++-- tools/perf/util/hist.h | 4 +- tools/perf/util/sort.c | 309 ++++++++++++++++++++++++++++++++++++- tools/perf/util/sort.h | 8 +- 14 files changed, 432 insertions(+), 114 deletions(-)
Signature · 1 line
2.6.4
Namhyung Kim
maintainerPATCH 1/6 perf hist: Pass struct sample to __hists__add_entry()
This is a preparation to add more info into the hist_entry. Also it already passes too many argument, so passing sample directly will reduce the overhead of the function call.
--- tools/perf/builtin-annotate.c | 7 +++++-- tools/perf/builtin-diff.c | 11 +++++------ tools/perf/tests/hists_link.c | 6 +++--- tools/perf/util/hist.c | 31 +++++++++++++++++-------------- tools/perf/util/hist.h | 4 ++-- 5 files changed, 32 insertions(+), 27 deletions(-)
tools/perf/builtin-annotate.c +5 -2
@@ -47,7 +47,7 @@ struct perf_annotate { };static int perf_evsel__add_sample (struct perf_evsel *evsel, -struct perf_sample *sample__maybe_unused , +struct perf_sample *sample,struct addr_location *al,struct perf_annotate *ann) {
@@ -72,7 +72,10 @@ static int perf_evsel__add_sample(struct perf_evsel *evsel,return 0 ; } - he =__hists__add_entry (hists, al,NULL ,NULL ,NULL ,1 ,1 ,0 , true); + sample->period =1 ; + sample->weight =1 ; + + he =__hists__add_entry (hists, al,NULL ,NULL ,NULL , sample, true);if (he ==NULL )return -ENOMEM ;
tools/perf/builtin-diff.c +5 -6
@@ -311,11 +311,11 @@ static int formula_fprintf(struct hist_entry *he, struct hist_entry *pair, }static int hists__add_entry (struct hists *hists, -struct addr_location *al,u64 period, -u64 weight,u64 transaction) +struct addr_location *al, +struct perf_sample *sample) { -if (__hists__add_entry (hists, al,NULL ,NULL ,NULL , period, weight, - transaction, true) !=NULL ) +if (__hists__add_entry (hists, al,NULL ,NULL ,NULL , + sample, true) !=NULL )return 0 ;return -ENOMEM ; }
@@ -336,8 +336,7 @@ static int diff__process_sample_event(struct perf_tool *tool __maybe_unused,return -1 ; } -if (hists__add_entry (hists, &al, sample->period, - sample->weight, sample->transaction)) { +if (hists__add_entry (hists, &al, sample)) {pr_warning ("problem incrementing symbol period, skipping event\n" ); goto out_put; }
tools/perf/tests/hists_link.c +3 -3
@@ -64,7 +64,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine)struct perf_evsel *evsel;struct addr_location al;struct hist_entry *he; -struct perf_sample sample = { .period =1 , }; +struct perf_sample sample = { .period =1 , .weight =1 , };size_t i =0 , k;/*
@@ -90,7 +90,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) goto out; he =__hists__add_entry (hists, &al,NULL , -NULL ,NULL ,1 ,1 ,0 , true); +NULL ,NULL , &sample, true);if (he ==NULL ) {addr_location__put (&al); goto out;
@@ -116,7 +116,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) goto out; he =__hists__add_entry (hists, &al,NULL , -NULL ,NULL ,1 ,1 ,0 , true); +NULL ,NULL , &sample, true);if (he ==NULL ) {addr_location__put (&al); goto out;
tools/perf/util/hist.c +17 -14
@@ -461,7 +461,7 @@ struct hist_entry *__hists__add_entry(struct hists *hists, struct symbol *sym_parent, struct branch_info *bi, struct mem_info *mi, - u64 period, u64 weight, u64 transaction, + struct perf_sample *sample, bool sample_self) { struct hist_entry entry = {
@@ -478,15 +478,15 @@ struct hist_entry *__hists__add_entry(struct hists *hists, .level = al->level, .stat = { .nr_events = 1, - .period = period, - .weight = weight, + .period = sample->period, + .weight = sample->weight, }, .parent = sym_parent, .filtered = symbol__parent_filter(sym_parent) | al->filtered, .hists = hists, .branch_info = bi, .mem_info = mi, - .transaction = transaction, + .transaction = sample->transaction, };return hists__findnew_entry (hists, &entry, al, sample_self);
@@ -526,12 +526,13 @@ iter_add_single_mem_entry(struct hist_entry_iter *iter, struct addr_location *alu64 cost;struct mem_info *mi = iter->priv;struct hists *hists =evsel__hists (iter->evsel); +struct perf_sample *sample = iter->sample;struct hist_entry *he;if (mi ==NULL )return -EINVAL ; - cost = iter->sample->weight; + cost = sample->weight;if (!cost) cost =1 ;
@@ -542,8 +543,10 @@ iter_add_single_mem_entry(struct hist_entry_iter *iter, struct addr_location *al* and this is indirectly achieved by passing period=weight here * and the he_stat__add_period() function. */ + sample->period = cost; + he =__hists__add_entry (hists, al, iter->parent,NULL , mi, - cost, cost,0 , true); + sample, true);if (!he)return -ENOMEM ;
@@ -630,6 +633,7 @@ iter_add_next_branch_entry(struct hist_entry_iter *iter, struct addr_location *astruct branch_info *bi;struct perf_evsel *evsel = iter->evsel;struct hists *hists =evsel__hists (evsel); +struct perf_sample *sample = iter->sample;struct hist_entry *he =NULL ;int i = iter->curr;int err =0 ;
@@ -643,9 +647,11 @@ iter_add_next_branch_entry(struct hist_entry_iter *iter, struct addr_location *a* The report shows the percentage of total branches captured * and not events sampled. Thus we use a pseudo period of 1. */ + sample->period =1 ; + sample->weight = bi->flags.cycles ? bi->flags.cycles :1 ; + he =__hists__add_entry (hists, al, iter->parent, &bi[i],NULL , -1 , bi->flags.cycles ? bi->flags.cycles :1 , -0 , true); + sample, true);if (he ==NULL )return -ENOMEM ;
@@ -682,8 +688,7 @@ iter_add_single_normal_entry(struct hist_entry_iter *iter, struct addr_locationstruct hist_entry *he; he =__hists__add_entry (evsel__hists (evsel), al, iter->parent,NULL ,NULL , - sample->period, sample->weight, - sample->transaction, true); + sample, true);if (he ==NULL )return -ENOMEM ;
@@ -744,8 +749,7 @@ iter_add_single_cumulative_entry(struct hist_entry_iter *iter,int err =0 ; he =__hists__add_entry (hists, al, iter->parent,NULL ,NULL , - sample->period, sample->weight, - sample->transaction, true); + sample, true);if (he ==NULL )return -ENOMEM ;
@@ -818,8 +822,7 @@ iter_add_next_cumulative_entry(struct hist_entry_iter *iter, } he =__hists__add_entry (evsel__hists (evsel), al, iter->parent,NULL ,NULL , - sample->period, sample->weight, - sample->transaction, false); + sample, false);if (he ==NULL )return -ENOMEM ;
tools/perf/util/hist.h +2 -2
@@ -114,8 +114,8 @@ struct hist_entry *__hists__add_entry(struct hists *hists,struct addr_location *al, struct symbol *parent, struct branch_info *bi, - struct mem_info *mi, u64 period, - u64 weight, u64 transaction, + struct mem_info *mi, + struct perf_sample *sample, bool sample_self);int hist_entry_iter__add (struct hist_entry_iter *iter,struct addr_location *al,int max_stack_depth,void *arg);
Signature · 1 line
2.6.4
Namhyung Kim
maintainerPATCH 2/6 perf hist: Save raw_data/size for tracepoint events
In reply to Namhyung KimThe raw_data and raw_size fields are to provide tracepoint specific information. They will be used by dynamic sort keys later.
--- tools/perf/util/hist.c | 2 ++ tools/perf/util/sort.h | 2 ++ 2 files changed, 4 insertions(+)
tools/perf/util/hist.c +2 -0
@@ -487,6 +487,8 @@ struct hist_entry *__hists__add_entry(struct hists *hists, .branch_info = bi, .mem_info = mi, .transaction = sample->transaction, + .raw_data = sample->raw_data, + .raw_size = sample->raw_size, };return hists__findnew_entry (hists, &entry, al, sample_self);
tools/perf/util/sort.h +2 -0
@@ -122,6 +122,8 @@ struct hist_entry {struct branch_info *branch_info;struct hists *hists;struct mem_info *mem_info; +void *raw_data; +u32 raw_size;struct callchain_root callchain[0 ];/* must be last member */ };
Signature · 1 line
2.6.4
Namhyung Kim
maintainerPATCH 3/6 tools lib traceevent: Factor out and export print_event_field()
In reply to Namhyung KimThe print_event_field() is to print basic information of a given field without the print format. It'll be used by dynamic sort keys later.
--- tools/lib/traceevent/event-parse.c | 124 ++++++++++++++++++++----------------- tools/lib/traceevent/event-parse.h | 2 + 2 files changed, 68 insertions(+), 58 deletions(-)
tools/lib/traceevent/event-parse.c +66 -58
@@ -4735,73 +4735,81 @@ static int is_printable_array(char *p, unsigned int len)return 1 ; } +void print_event_field (struct trace_seq *s,void *data, +struct format_field *field) +{ +unsigned long long val; +unsigned int offset, len, i; +struct pevent *pevent = field->event->pevent; + +if (field->flags &FIELD_IS_ARRAY ) { + offset = field->offset; + len = field->size; +if (field->flags &FIELD_IS_DYNAMIC ) { + val =pevent_read_number (pevent, data + offset, len); + offset = val; + len = offset >>16 ; + offset &=0xffff ; + } +if (field->flags &FIELD_IS_STRING && +is_printable_array (data + offset, len)) { +trace_seq_printf (s,"%s" , (char *)data + offset); + }else { +trace_seq_puts (s,"ARRAY[" ); +for (i =0 ; i < len; i++) { +if (i) +trace_seq_puts (s,", " ); +trace_seq_printf (s,"%02x" , + *((unsigned char *)data + offset + i)); + } +trace_seq_putc (s,']' ); + field->flags &= ~FIELD_IS_STRING ; + } + }else { + val =pevent_read_number (pevent, data + field->offset, + field->size); +if (field->flags &FIELD_IS_POINTER ) { +trace_seq_printf (s,"0x%llx" , val); + }else if (field->flags &FIELD_IS_SIGNED ) { +switch (field->size) { +case 4 : +/* +* If field is long then print it in hex. +* A long usually stores pointers. +*/ +if (field->flags &FIELD_IS_LONG ) +trace_seq_printf (s,"0x%x" , (int )val); +else +trace_seq_printf (s,"%d" , (int )val); +break ; +case 2 : +trace_seq_printf (s,"%2d" , (short )val); +break ; +case 1 : +trace_seq_printf (s,"%1d" , (char )val); +break ; +default : +trace_seq_printf (s,"%lld" , val); + } + }else { +if (field->flags &FIELD_IS_LONG ) +trace_seq_printf (s,"0x%llx" , val); +else +trace_seq_printf (s,"%llu" , val); + } + } +} +static void print_event_fields (struct trace_seq *s,void *data,int size__maybe_unused ,struct event_format *event) {struct format_field *field; -unsigned long long val; -unsigned int offset, len, i; field = event->format.fields;while (field) {trace_seq_printf (s," %s=" , field->name); -if (field->flags &FIELD_IS_ARRAY ) { - offset = field->offset; - len = field->size; -if (field->flags &FIELD_IS_DYNAMIC ) { - val =pevent_read_number (event->pevent, data + offset, len); - offset = val; - len = offset >>16 ; - offset &=0xffff ; - } -if (field->flags &FIELD_IS_STRING && -is_printable_array (data + offset, len)) { -trace_seq_printf (s,"%s" , (char *)data + offset); - }else { -trace_seq_puts (s,"ARRAY[" ); -for (i =0 ; i < len; i++) { -if (i) -trace_seq_puts (s,", " ); -trace_seq_printf (s,"%02x" , - *((unsigned char *)data + offset + i)); - } -trace_seq_putc (s,']' ); - field->flags &= ~FIELD_IS_STRING ; - } - }else { - val =pevent_read_number (event->pevent, data + field->offset, - field->size); -if (field->flags &FIELD_IS_POINTER ) { -trace_seq_printf (s,"0x%llx" , val); - }else if (field->flags &FIELD_IS_SIGNED ) { -switch (field->size) { -case 4 : -/* -* If field is long then print it in hex. -* A long usually stores pointers. -*/ -if (field->flags &FIELD_IS_LONG ) -trace_seq_printf (s,"0x%x" , (int )val); -else -trace_seq_printf (s,"%d" , (int )val); -break ; -case 2 : -trace_seq_printf (s,"%2d" , (short )val); -break ; -case 1 : -trace_seq_printf (s,"%1d" , (char )val); -break ; -default : -trace_seq_printf (s,"%lld" , val); - } - }else { -if (field->flags &FIELD_IS_LONG ) -trace_seq_printf (s,"0x%llx" , val); -else -trace_seq_printf (s,"%llu" , val); - } - } +print_event_field (s, data, field); field = field->next; } }
tools/lib/traceevent/event-parse.h +2 -0
@@ -705,6 +705,8 @@ struct cmdline *pevent_data_pid_from_comm(struct pevent *pevent, const char *com struct cmdline *next);int pevent_cmdline_pid (struct pevent *pevent,struct cmdline *cmdline); +void print_event_field (struct trace_seq *s,void *data, +struct format_field *field);void pevent_event_info (struct trace_seq *s,struct event_format *event,struct pevent_record *record); int pevent_strerror(struct pevent *pevent, enum pevent_errno errnum,
Signature · 1 line
2.6.4
Namhyung Kim
maintainerPATCH 4/6 perf tools: Pass evlist to setup_sorting()
In reply to Namhyung KimThis is a preparation to support dynamic sort keys for tracepoint events. Dynamic sort keys can be created for specific fields in trace events so it needs the event information.
--- tools/perf/builtin-annotate.c | 2 +- tools/perf/builtin-diff.c | 2 +- tools/perf/builtin-report.c | 2 +- tools/perf/builtin-top.c | 14 +++++++------- tools/perf/tests/hists_cumulate.c | 8 ++++---- tools/perf/tests/hists_filter.c | 2 +- tools/perf/tests/hists_link.c | 2 +- tools/perf/tests/hists_output.c | 10 +++++----- tools/perf/util/sort.c | 15 +++++++++------ tools/perf/util/sort.h | 5 +++-- 10 files changed, 33 insertions(+), 29 deletions(-)
tools/perf/builtin-annotate.c +1 -1
@@ -370,7 +370,7 @@ int cmd_annotate(int argc, const char **argv, const char *prefix __maybe_unused)if (ret <0 ) goto out_delete; -if (setup_sorting () <0 ) +if (setup_sorting (NULL ) <0 )usage_with_options (annotate_usage, options);if (annotate.use_stdio)
tools/perf/builtin-diff.c +1 -1
@@ -1279,7 +1279,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix __maybe_unused) sort__mode =SORT_MODE__DIFF ; -if (setup_sorting () <0 ) +if (setup_sorting (NULL ) <0 )usage_with_options (diff_usage, options);setup_pager ();
tools/perf/builtin-report.c +1 -1
@@ -897,7 +897,7 @@ repeat: symbol_conf.cumulate_callchain = false; } -if (setup_sorting () <0 ) { +if (setup_sorting (session->evlist) <0 ) {if (sort_order)parse_options_usage (report_usage, options,"s" ,1 );if (field_order)
tools/perf/builtin-top.c +7 -7
@@ -1231,11 +1231,17 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)if (argc)usage_with_options (top_usage, options); +if (!top.evlist->nr_entries && +perf_evlist__add_default (top.evlist) <0 ) { +pr_err ("Not enough memory for event selector list\n" ); + goto out_delete_evlist; + } + sort__mode =SORT_MODE__TOP ;/* display thread wants entries to be collapsed in a different tree */ sort__need_collapse =1 ; -if (setup_sorting () <0 ) { +if (setup_sorting (top.evlist) <0 ) {if (sort_order)parse_options_usage (top_usage, options,"s" ,1 );if (field_order)
@@ -1277,12 +1283,6 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused) goto out_delete_evlist; } -if (!top.evlist->nr_entries && -perf_evlist__add_default (top.evlist) <0 ) { -ui__error ("Not enough memory for event selector list\n" ); - goto out_delete_evlist; - } - symbol_conf.nr_events = top.evlist->nr_entries;if (top.delay_secs <1 )
tools/perf/tests/hists_cumulate.c +4 -4
@@ -281,7 +281,7 @@ static int test1(struct perf_evsel *evsel, struct machine *machine) symbol_conf.cumulate_callchain = false;perf_evsel__reset_sample_bit (evsel,CALLCHAIN ); -setup_sorting (); +setup_sorting (NULL );callchain_register_param (&callchain_param); err =add_hist_entries (hists, machine);
@@ -428,7 +428,7 @@ static int test2(struct perf_evsel *evsel, struct machine *machine) symbol_conf.cumulate_callchain = false;perf_evsel__set_sample_bit (evsel,CALLCHAIN ); -setup_sorting (); +setup_sorting (NULL );callchain_register_param (&callchain_param); err =add_hist_entries (hists, machine);
@@ -486,7 +486,7 @@ static int test3(struct perf_evsel *evsel, struct machine *machine) symbol_conf.cumulate_callchain = true;perf_evsel__reset_sample_bit (evsel,CALLCHAIN ); -setup_sorting (); +setup_sorting (NULL );callchain_register_param (&callchain_param); err =add_hist_entries (hists, machine);
@@ -670,7 +670,7 @@ static int test4(struct perf_evsel *evsel, struct machine *machine) symbol_conf.cumulate_callchain = true;perf_evsel__set_sample_bit (evsel,CALLCHAIN ); -setup_sorting (); +setup_sorting (NULL );callchain_register_param (&callchain_param); err =add_hist_entries (hists, machine);
tools/perf/tests/hists_filter.c +1 -1
@@ -122,7 +122,7 @@ int test__hists_filter(int subtest __maybe_unused) goto out;/* default sort order (comm,dso,sym) will be used */ -if (setup_sorting () <0 ) +if (setup_sorting (NULL ) <0 ) goto out;machines__init (&machines);
tools/perf/tests/hists_link.c +1 -1
@@ -294,7 +294,7 @@ int test__hists_link(int subtest __maybe_unused) goto out;/* default sort order (comm,dso,sym) will be used */ -if (setup_sorting () <0 ) +if (setup_sorting (NULL ) <0 ) goto out;machines__init (&machines);
tools/perf/tests/hists_output.c +5 -5
@@ -134,7 +134,7 @@ static int test1(struct perf_evsel *evsel, struct machine *machine) field_order =NULL ; sort_order =NULL ;/* equivalent to sort_order = "comm,dso,sym" */ -setup_sorting (); +setup_sorting (NULL );/* * expected output:
@@ -236,7 +236,7 @@ static int test2(struct perf_evsel *evsel, struct machine *machine) field_order ="overhead,cpu" ; sort_order ="pid" ; -setup_sorting (); +setup_sorting (NULL );/* * expected output:
@@ -292,7 +292,7 @@ static int test3(struct perf_evsel *evsel, struct machine *machine) field_order ="comm,overhead,dso" ; sort_order =NULL ; -setup_sorting (); +setup_sorting (NULL );/* * expected output:
@@ -366,7 +366,7 @@ static int test4(struct perf_evsel *evsel, struct machine *machine) field_order ="dso,sym,comm,overhead,dso" ; sort_order ="sym" ; -setup_sorting (); +setup_sorting (NULL );/* * expected output:
@@ -468,7 +468,7 @@ static int test5(struct perf_evsel *evsel, struct machine *machine) field_order ="cpu,pid,comm,dso,sym" ; sort_order ="dso,pid" ; -setup_sorting (); +setup_sorting (NULL );/* * expected output:
tools/perf/util/sort.c +9 -6
@@ -4,6 +4,8 @@#include "comm.h" #include "symbol.h" #include "evsel.h" +#include "evlist.h" +#include <traceevent/event-parse.h> regex_t parent_regex;const char default_parent_pattern[] ="^sys_|^do_page_fault" ;
@@ -1583,7 +1585,8 @@ int hpp_dimension__add_output(unsigned col)return __hpp_dimension__add_output (&hpp_sort_dimensions[col]); } -int sort_dimension__add (const char *tok) +static int sort_dimension__add (const char *tok, +struct perf_evlist *evlist__maybe_unused ) {unsigned int i;
@@ -1712,7 +1715,7 @@ static int setup_sort_order(void)return 0 ; } -static int __setup_sorting (void ) +static int __setup_sorting (struct perf_evlist *evlist) {char *tmp, *tok, *str;const char *sort_keys;
@@ -1743,7 +1746,7 @@ static int __setup_sorting(void)for (tok =strtok_r (str,", " , &tmp); tok; tok =strtok_r (NULL ,", " , &tmp)) { - ret =sort_dimension__add (tok); + ret =sort_dimension__add (tok, evlist);if (ret == -EINVAL ) {error ("Invalid --sort key: `%s'" , tok);break ;
@@ -1954,16 +1957,16 @@ out:return ret; } -int setup_sorting (void ) +int setup_sorting (struct perf_evlist *evlist) {int err; - err =__setup_sorting (); + err =__setup_sorting (evlist);if (err <0 )return err;if (parent_pattern != default_parent_pattern) { - err =sort_dimension__add ("parent" ); + err =sort_dimension__add ("parent" , evlist);if (err <0 )return err; }
tools/perf/util/sort.h +3 -2
@@ -226,10 +226,11 @@ struct sort_entry {extern struct sort_entry sort_thread;extern struct list_head hist_entry__sort_list; -int setup_sorting (void ); +struct perf_evlist ; +struct pevent ; +int setup_sorting (struct perf_evlist *evlist);int setup_output_field (void );void reset_output_field (void ); -extern int sort_dimension__add (const char *);void sort__setup_elide (FILE *fp);void perf_hpp__set_elide (int idx,bool elide);
Signature · 1 line
2.6.4
Namhyung Kim
maintainerPATCH 5/6 perf tools: Add dynamic sort key for tracepoint events
In reply to Namhyung KimThe existing sort keys are less useful for tracepoint events in that they are always sampled at a same location.
For example, report on sched:sched_switch event looks like following
# Overhead Command Shared Object Symbol
# ........ ............... ................ ..............
#
47.22% swapper [kernel.vmlinux] [k] __schedule
21.67% transmission-gt [kernel.vmlinux] [k] __schedule
8.23% netctl-auto [kernel.vmlinux] [k] __schedule
5.53% kworker/0:1H [kernel.vmlinux] [k] __schedule
1.98% Xephyr [kernel.vmlinux] [k] __schedule
1.33% irq/33-iwlwifi [kernel.vmlinux] [k] __schedule
1.17% wpa_cli [kernel.vmlinux] [k] __schedule
1.13% rcu_preempt [kernel.vmlinux] [k] __schedule
0.85% ksoftirqd/0 [kernel.vmlinux] [k] __schedule
0.77% Timer [kernel.vmlinux] [k] __scheduleIn fact, tracepoints have meaningful information in their fields but there's no way to use in the perf report currently. The dynamic sort keys are to overcome this problem.
The sched:sched_switch events have following fields:
# sudo cat /sys/kernel/debug/tracing/events/sched/sched_switch/format name: sched_switch ID: 268 format: field:unsigned short common_type; offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2; size:1; signed:0; field:unsigned char common_preempt_count; offset:3; size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1;
field:char prev_comm[16]; offset:8; size:16; signed:1; field:pid_t prev_pid; offset:24; size:4; signed:1; field:int prev_prio; offset:28; size:4; signed:1; field:long prev_state; offset:32; size:8; signed:1; field:char next_comm[16]; offset:40; size:16; signed:1; field:pid_t next_pid; offset:56; size:4; signed:1; field:int next_prio; offset:60; size:4; signed:1;
print fmt: "prev_comm=%s prev_pid=%d prev_prio=%d prev_state=%s%s ==>
next_comm=%s next_pid=%d next_prio=%d",
REC->prev_comm, REC->prev_pid, REC->prev_prio,
REC->prev_state & (2048-1) ? __print_flags(REC->prev_state & (2048-1),
"|", { 1, "S"} , { 2, "D" }, { 4, "T" }, { 8, "t" }, { 16, "Z" }, { 32, "X" },
{ 64, "x" }, { 128, "K"}, { 256, "W" }, { 512, "P" }, { 1024, "N" }) : "R",
REC->prev_state & 2048 ? "+" : "", REC->next_comm, REC->next_pid, REC->next_prioWith dynamic sort keys, you can use <event.field> as a sort key. Those dynamic keys are checked and created on demand. For instance, below is to sort by next_pid field on the same data file.
$ perf report -s comm,sched:sched_switch.next_pid --stdio
...
# Overhead Command next_pid
# ........ ............... ..........
#
21.23% transmission-gt 0
20.86% swapper 17773
6.62% netctl-auto 0
5.25% swapper 109
5.21% kworker/0:1H 0
1.98% Xephyr 0
1.98% swapper 6524
1.98% swapper 27478
1.37% swapper 27476
1.17% swapper 233Multiple dynamic sort keys are also supported:
$ perf report -s comm,sched:sched_switch.next_pid,sched:sched_switch.next_comm --stdio
...
# Overhead Command next_pid next_comm
# ........ ............... .......... ................
#
20.86% swapper 17773 transmission-gt
9.64% transmission-gt 0 swapper/0
9.16% transmission-gt 0 swapper/2
5.25% swapper 109 kworker/0:1H
5.21% kworker/0:1H 0 swapper/0
2.14% netctl-auto 0 swapper/2
1.98% netctl-auto 0 swapper/0
1.98% swapper 6524 Xephyr
1.98% swapper 27478 netctl-auto
1.78% transmission-gt 0 swapper/3
1.53% Xephyr 0 swapper/0
1.29% netctl-auto 0 swapper/1
1.29% swapper 27476 netctl-auto
1.21% netctl-auto 0 swapper/3
1.17% swapper 233 irq/33-iwlwifiNote that pid 0 exists for each cpu so have comm of 'swapper/N'.
--- tools/perf/util/sort.c | 223 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 223 insertions(+)
tools/perf/util/sort.c +223 -0
@@ -1531,6 +1531,226 @@ static int __sort_dimension__add_hpp_output(struct sort_dimension *sd)return 0 ; } +struct hpp_dynamic_entry { +struct perf_hpp_fmt hpp; +struct perf_evsel *evsel; +struct format_field *field; +unsigned dynamic_len; +}; + +static int hde_width (struct hpp_dynamic_entry *hde) +{ +if (!hde->hpp.len) { +int len = hde->dynamic_len; +int namelen =strlen (hde->field->name); +int fieldlen = hde->field->size; + +if (namelen > len) + len = namelen; + +if (!(hde->field->flags &FIELD_IS_STRING )) { +/* length for print hex numbers */ + fieldlen = hde->field->size *2 +2 ; + } +if (fieldlen > len) + len = fieldlen; + + hde->hpp.len = len; + } +return hde->hpp.len; +} + +static int __sort__hde_header (struct perf_hpp_fmt *fmt,struct perf_hpp *hpp, +struct perf_evsel *evsel__maybe_unused ) +{ +struct hpp_dynamic_entry *hde; +size_t len = fmt->user_len; + + hde = container_of(fmt, struct hpp_dynamic_entry, hpp); + +if (!len) + len =hde_width (hde); + +return scnprintf (hpp->buf, hpp->size,"%*.*s" , len, len, hde->field->name); +} + +static int __sort__hde_width (struct perf_hpp_fmt *fmt, +struct perf_hpp *hpp__maybe_unused , +struct perf_evsel *evsel__maybe_unused ) +{ +struct hpp_dynamic_entry *hde; +size_t len = fmt->user_len; + + hde = container_of(fmt, struct hpp_dynamic_entry, hpp); + +if (!len) + len =hde_width (hde); + +return len; +} + +static int __sort__hde_entry (struct perf_hpp_fmt *fmt,struct perf_hpp *hpp, +struct hist_entry *he) +{ +struct hpp_dynamic_entry *hde; +size_t len = fmt->user_len; +struct trace_seq seq; +int ret; + + hde = container_of(fmt, struct hpp_dynamic_entry, hpp); + +if (!len) + len =hde_width (hde); + +if (hists_to_evsel (he->hists) != hde->evsel) +return scnprintf (hpp->buf, hpp->size,"%*.*s" , len, len,"N/A" ); + +trace_seq_init (&seq); +print_event_field (&seq, he->raw_data, hde->field); + ret =scnprintf (hpp->buf, hpp->size,"%*.*s" , len, len, seq.buffer); +trace_seq_destroy (&seq); +return ret; +} + +static int64_t __sort__hde_cmp (struct perf_hpp_fmt *fmt, +struct hist_entry *a,struct hist_entry *b) +{ +struct hpp_dynamic_entry *hde; +struct format_field *field; +unsigned offset, size; + + hde = container_of(fmt, struct hpp_dynamic_entry, hpp); + +if (hists_to_evsel (a->hists) != hde->evsel) +return 0 ; + + field = hde->field; +if (field->flags &FIELD_IS_DYNAMIC ) { +unsigned long long dyn; + +pevent_read_number_field (field, a->raw_data, &dyn); + offset = dyn &0xffff ; + size = (dyn >>16 ) &0xffff ; + +/* record max width for output */ +if (size > hde->dynamic_len) + hde->dynamic_len = size; + }else { + offset = field->offset; + size = field->size; + } + +return memcmp (a->raw_data + offset, b->raw_data + offset, size); +} + +static struct hpp_dynamic_entry * +__alloc_dynamic_entry (struct perf_evsel *evsel,struct format_field *field) +{ +struct hpp_dynamic_entry *hde; + + hde =malloc (sizeof (*hde)); +if (hde ==NULL ) { +pr_err ("Memory allocation failed\n" ); +return NULL ; + } + + hde->evsel = evsel; + hde->field = field; + hde->dynamic_len =0 ; + + hde->hpp.name = field->name; + hde->hpp.header = __sort__hde_header; + hde->hpp.width = __sort__hde_width; + hde->hpp.entry = __sort__hde_entry; + hde->hpp.color =NULL ; + + hde->hpp.cmp = __sort__hde_cmp; + hde->hpp.collapse = __sort__hde_cmp; + hde->hpp.sort = __sort__hde_cmp; + +INIT_LIST_HEAD (&hde->hpp.list); +INIT_LIST_HEAD (&hde->hpp.sort_list); + hde->hpp.elide = false; + hde->hpp.len =0 ; + hde->hpp.user_len =0 ; + +return hde; +} + +static int add_dynamic_entry (struct perf_evlist *evlist,const char *tok) +{ +char *str, *event_name, *field_name; +struct perf_evsel *evsel, *pos; +struct event_format *format; +struct format_field *field; +struct hpp_dynamic_entry *hde; +int ret =0 ; + +if (evlist ==NULL ) +return -ENOENT ; + + str =strdup (tok); +if (str ==NULL ) +return -ENOMEM ; + + event_name = str; + field_name =strchr (str,'.' ); +if (field_name ==NULL ) { + ret = -EINVAL ; + goto out; + } + *field_name++ ='\0' ; + + evsel =NULL ; +evlist__for_each (evlist, pos) { +if (!strcmp (pos->name, event_name)) { + evsel = pos; +break ; + } + } + +if (evsel ==NULL ) { +pr_err ("Cannot find event: %s\n" , event_name); + ret = -ENOENT ; + goto out; + } + +if (evsel->attr.type !=PERF_TYPE_TRACEPOINT ) { +pr_err ("%s is not a tracepoint event\n" , event_name); + ret = -EINVAL ; + goto out; + } + + format =pevent_find_event (evsel->tp_format->pevent, + evsel->attr.config); +if (format ==NULL ) { +pr_err ("Cannot find event format for %s (id: %u)\n" , + event_name, (unsigned ) evsel->attr.config); + ret = -ENOENT ; + goto out; + } + + field =pevent_find_any_field (format, field_name); +if (field ==NULL ) { +pr_err ("Cannot find event field for %s.%s\n" , + event_name, field_name); + ret = -ENOENT ; + goto out; + } + + hde =__alloc_dynamic_entry (evsel, field); +if (hde ==NULL ) { + ret = -ENOMEM ; + goto out; + } + +perf_hpp__register_sort_field (&hde->hpp); + +out: +free (str); +return ret; +} +static int __sort_dimension__add (struct sort_dimension *sd) {if (sd->taken)
@@ -1667,6 +1887,9 @@ static int sort_dimension__add(const char *tok,return 0 ; } +if (!add_dynamic_entry (evlist, tok)) +return 0 ; +return -ESRCH ; }
Signature · 1 line
2.6.4
Re: [PATCH 5/6] perf tools: Add dynamic sort key for tracepoint events
On Tue, Dec 15, 2015 at 12:46:12AM +0900, Namhyung Kim wrote:
SNIP
Quoted 22 lines
$ perf report -s comm,sched:sched_switch.next_pid,sched:sched_switch.next_comm --stdio
...
# Overhead Command next_pid next_comm
# ........ ............... .......... ................
#
20.86% swapper 17773 transmission-gt
9.64% transmission-gt 0 swapper/0
9.16% transmission-gt 0 swapper/2
5.25% swapper 109 kworker/0:1H
5.21% kworker/0:1H 0 swapper/0
2.14% netctl-auto 0 swapper/2
1.98% netctl-auto 0 swapper/0
1.98% swapper 6524 Xephyr
1.98% swapper 27478 netctl-auto
1.78% transmission-gt 0 swapper/3
1.53% Xephyr 0 swapper/0
1.29% netctl-auto 0 swapper/1
1.29% swapper 27476 netctl-auto
1.21% netctl-auto 0 swapper/3
1.17% swapper 233 irq/33-iwlwifiQuoted 1 line
Note that pid 0 exists for each cpu so have comm of 'swapper/N'.
could we also add by default all tracepoint fields in case none is specified and the event to display is tracepoint?
also an extra field that would hold/show the 'print fmt' display ?
thanks,
jirka
Namhyung Kim
maintainerRe: [PATCH 5/6] perf tools: Add dynamic sort key for tracepoint events
Quoted 26 lines · Jiri Olsa
On Tue, Dec 15, 2015 at 12:46:12AM +0900, Namhyung Kim wrote:
SNIP
Quoted 22 lines
$ perf report -s comm,sched:sched_switch.next_pid,sched:sched_switch.next_comm --stdio ... # Overhead Command next_pid next_comm # ........ ............... .......... ................ # 20.86% swapper 17773 transmission-gt 9.64% transmission-gt 0 swapper/0 9.16% transmission-gt 0 swapper/2 5.25% swapper 109 kworker/0:1H 5.21% kworker/0:1H 0 swapper/0 2.14% netctl-auto 0 swapper/2 1.98% netctl-auto 0 swapper/0 1.98% swapper 6524 Xephyr 1.98% swapper 27478 netctl-auto 1.78% transmission-gt 0 swapper/3 1.53% Xephyr 0 swapper/0 1.29% netctl-auto 0 swapper/1 1.29% swapper 27476 netctl-auto 1.21% netctl-auto 0 swapper/3 1.17% swapper 233 irq/33-iwlwifi
Quoted 4 lines
Quoted 1 line
Note that pid 0 exists for each cpu so have comm of 'swapper/N'.
could we also add by default all tracepoint fields in case none is specified and the event to display is tracepoint?
Seems like a good suggestion. We can check if there's only one tracepoint event, then use dynamic sort keys for all fields. But I think we should skip common fields in that case.
Quoted 2 lines
also an extra field that would hold/show the 'print fmt' display ?
Do you want a single extra field per event or per field?
Thanks,
Namhyung
Re: [PATCH 5/6] perf tools: Add dynamic sort key for tracepoint events
Quoted 32 lines · Namhyung Kim
Quoted 26 lines · Jiri Olsa
On Tue, Dec 15, 2015 at 12:46:12AM +0900, Namhyung Kim wrote:
SNIP
Quoted 22 lines
$ perf report -s comm,sched:sched_switch.next_pid,sched:sched_switch.next_comm --stdio ... # Overhead Command next_pid next_comm # ........ ............... .......... ................ # 20.86% swapper 17773 transmission-gt 9.64% transmission-gt 0 swapper/0 9.16% transmission-gt 0 swapper/2 5.25% swapper 109 kworker/0:1H 5.21% kworker/0:1H 0 swapper/0 2.14% netctl-auto 0 swapper/2 1.98% netctl-auto 0 swapper/0 1.98% swapper 6524 Xephyr 1.98% swapper 27478 netctl-auto 1.78% transmission-gt 0 swapper/3 1.53% Xephyr 0 swapper/0 1.29% netctl-auto 0 swapper/1 1.29% swapper 27476 netctl-auto 1.21% netctl-auto 0 swapper/3 1.17% swapper 233 irq/33-iwlwifiQuoted 4 lines
Quoted 1 line
Note that pid 0 exists for each cpu so have comm of 'swapper/N'.
could we also add by default all tracepoint fields in case none is specified and the event to display is tracepoint?
Quoted 8 lines
Seems like a good suggestion. We can check if there's only one tracepoint event, then use dynamic sort keys for all fields. But I think we should skip common fields in that case.
Quoted 2 lines
also an extra field that would hold/show the 'print fmt' display ?
Do you want a single extra field per event or per field?
hm, so the 'print fmt' defines the intended output from the tracepoint, like for sched_switch:
print fmt: "prev_comm=%s prev_pid=%d prev_prio=%d prev_state=%s%s ==> next_comm=%s next_pid=%d next_prio=%d", REC->prev_comm, REC->prev_pid, REC->prev_prio, REC->prev_state & (2048-1) ? __print_flags(REC->prev_state & (2048-1), "|", { 1, "S"} , { 2, "D" }, { 4, "T" }, { 8, "t" }, { 16, "Z" }, { 32, "X" }, { 64, "x" }, { 128, "K" }, { 256, "W" }, { 512, "P" }, { 1024, "N" }) : "R", REC->prev_state & 2048 ? "+" : "", REC->next_comm, REC->next_pid, REC->next_prio
gets you (perf script can already do that):
perf:21226 [120] S ==> swapper/0:0 [120]
so maybe have a option or ahve a special field like 'fmt' that would carry/display this translation
perf report -s comm,fmt
could be combined with other fields if needed..
jirka
Namhyung Kim
maintainerRe: [PATCH 5/6] perf tools: Add dynamic sort key for tracepoint events
Quoted 52 lines · Jiri Olsa
Quoted 32 lines · Namhyung Kim
Quoted 26 lines · Jiri Olsa
On Tue, Dec 15, 2015 at 12:46:12AM +0900, Namhyung Kim wrote:
SNIP
Quoted 22 lines
$ perf report -s comm,sched:sched_switch.next_pid,sched:sched_switch.next_comm --stdio ... # Overhead Command next_pid next_comm # ........ ............... .......... ................ # 20.86% swapper 17773 transmission-gt 9.64% transmission-gt 0 swapper/0 9.16% transmission-gt 0 swapper/2 5.25% swapper 109 kworker/0:1H 5.21% kworker/0:1H 0 swapper/0 2.14% netctl-auto 0 swapper/2 1.98% netctl-auto 0 swapper/0 1.98% swapper 6524 Xephyr 1.98% swapper 27478 netctl-auto 1.78% transmission-gt 0 swapper/3 1.53% Xephyr 0 swapper/0 1.29% netctl-auto 0 swapper/1 1.29% swapper 27476 netctl-auto 1.21% netctl-auto 0 swapper/3 1.17% swapper 233 irq/33-iwlwifiQuoted 4 lines
Quoted 1 line
Note that pid 0 exists for each cpu so have comm of 'swapper/N'.
could we also add by default all tracepoint fields in case none is specified and the event to display is tracepoint?
Quoted 8 lines
Seems like a good suggestion. We can check if there's only one tracepoint event, then use dynamic sort keys for all fields. But I think we should skip common fields in that case.
Quoted 2 lines
also an extra field that would hold/show the 'print fmt' display ?
Do you want a single extra field per event or per field?
hm, so the 'print fmt' defines the intended output from the tracepoint, like for
sched_switch:print fmt: "prev_comm=%s prev_pid=%d prev_prio=%d prev_state=%s%s ==> next_comm=%s next_pid=%d next_prio=%d",
REC->prev_comm,REC->prev_pid,REC->prev_prio,REC->prev_state& (2048-1) ? __print_flags(REC->prev_state & (2048-1), "|", { 1, "S"} , { 2, "D" }, { 4, "T" }, { 8, "t" }, { 16, "Z" }, { 32, "X" }, { 64, "x" }, { 128, "K" }, { 256, "W" }, { 512, "P" }, { 1024, "N" }) : "R",REC->prev_state& 2048 ? "+" : "",REC->next_comm,REC->next_pid,REC->next_priogets you (perf script can already do that):
perf:21226 [120] S ==> swapper/0:0 [120]
Quoted 6 lines
so maybe have a option or ahve a special field like 'fmt' that would carry/display this translation
perf report -s comm,fmt
could be combined with other fields if needed..
OK. I'll try to implement the 'fmt' sort key and use it for tracepoint events by default.
Thanks,
Namhyung
Namhyung Kim
maintainerPATCH 6/6 perf tools: Try to show pretty printed output for dynamic sort keys
In reply to Namhyung KimEach tracepoint event has format string for print to improve readability. Try to parse the output and match the field name. If it finds one, use that for the result. If not, fallbacks to the original output.
For example, sort on kmem:kmalloc.gfp_flags looks like below:
(Note: libtraceevent plugins are not installed on my system. They might affect the output below)
Before:
# Overhead Command gfp_flags
# ........ ....... ..........
#
99.89% perf 32848
0.06% sleep 208
0.03% perf 32976
0.01% perf 208After:
# Overhead Command gfp_flags
# ........ ....... ...................
#
99.89% perf GFP_NOFS|GFP_ZERO
0.06% sleep GFP_KERNEL
0.03% perf GFP_KERNEL|GFP_ZERO
0.01% perf GFP_KERNEL--- tools/perf/util/sort.c | 75 ++++++++++++++++++++++++++++++++++++++++++++++++-- tools/perf/util/sort.h | 1 + 2 files changed, 74 insertions(+), 2 deletions(-)
tools/perf/util/sort.c +73 -2
@@ -1560,6 +1560,47 @@ static int hde_width(struct hpp_dynamic_entry *hde)return hde->hpp.len; } +static void update_dynamic_len (struct hpp_dynamic_entry *hde, +struct hist_entry *he) +{ +char *str, *pos; +struct trace_seq seq; +struct format_field *field = hde->field; +struct pevent_record rec = { + .cpu = he->cpu, + .data = he->raw_data, + .size = he->raw_size, + }; +size_t namelen; + +if (he->dynlen_updated) +return ; + +/* parse pretty print result and update max length */ +trace_seq_init (&seq); +pevent_event_info (&seq, field->event, &rec); + + namelen =strlen (field->name); + str =strtok_r (seq.buffer," " , &pos); +while (str) { +if (!strncmp (str, field->name, namelen)) { +size_t len; + + str += namelen +1 ; + len =strlen (str); + +if (len > hde->dynamic_len) + hde->dynamic_len = len; +break ; + } + + str =strtok_r (NULL ," " , &pos); + } +trace_seq_destroy (&seq); + + he->dynlen_updated = true; +} +static int __sort__hde_header (struct perf_hpp_fmt *fmt,struct perf_hpp *hpp,struct perf_evsel *evsel__maybe_unused ) {
@@ -1595,6 +1636,14 @@ static int __sort__hde_entry(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,struct hpp_dynamic_entry *hde;size_t len = fmt->user_len;struct trace_seq seq; +char *str, *pos; +struct format_field *field; +struct pevent_record rec = { + .cpu = he->cpu, + .data = he->raw_data, + .size = he->raw_size, + }; +size_t namelen;int ret; hde = container_of(fmt, struct hpp_dynamic_entry, hpp);
@@ -1605,9 +1654,28 @@ static int __sort__hde_entry(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,if (hists_to_evsel (he->hists) != hde->evsel)return scnprintf (hpp->buf, hpp->size,"%*.*s" , len, len,"N/A" ); + field = hde->field;trace_seq_init (&seq); -print_event_field (&seq, he->raw_data, hde->field); - ret =scnprintf (hpp->buf, hpp->size,"%*.*s" , len, len, seq.buffer); +pevent_event_info (&seq, field->event, &rec); + + namelen =strlen (field->name); + str =strtok_r (seq.buffer," " , &pos); +while (str) { +if (!strncmp (str, field->name, namelen)) { + str += namelen +1 ; +break ; + } + + str =strtok_r (NULL ," " , &pos); + } + +if (str ==NULL ) { +trace_seq_reset (&seq); +print_event_field (&seq, he->raw_data, hde->field); + str = seq.buffer; + } + + ret =scnprintf (hpp->buf, hpp->size,"%*.*s" , len, len, str);trace_seq_destroy (&seq);return ret; }
@@ -1638,6 +1706,9 @@ static int64_t __sort__hde_cmp(struct perf_hpp_fmt *fmt, }else { offset = field->offset; size = field->size; + +update_dynamic_len (hde, a); +update_dynamic_len (hde, b); }return memcmp (a->raw_data + offset, b->raw_data + offset, size);
tools/perf/util/sort.h +1 -0
@@ -97,6 +97,7 @@ struct hist_entry {/* We are added by hists__add_dummy_entry. */ bool dummy; +bool dynlen_updated;char level;u8 filtered;
Signature · 1 line
2.6.4
Re: [PATCH 6/6] perf tools: Try to show pretty printed output for dynamic sort keys
Quoted 18 lines · Namhyung Kim
Each tracepoint event has format string for print to improve readability. Try to parse the output and match the field name. If it finds one, use that for the result. If not, fallbacks to the original output.
For example, sort on kmem:kmalloc.gfp_flags looks like below:
(Note: libtraceevent plugins are not installed on my system. They might affect the output below)Before:
# Overhead Commandgfp_flags# ........ ....... .......... # 99.89% perf 32848 0.06% sleep 208 0.03% perf 32976 0.01% perf 208
Quoted 8 lines
After:
# Overhead Commandgfp_flags# ........ ....... ................... # 99.89% perf GFP_NOFS|GFP_ZERO 0.06% sleep GFP_KERNEL 0.03% perf GFP_KERNEL|GFP_ZERO 0.01% perf GFP_KERNEL
hum, maybe we want some way to switch back to numbers?
jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 6/6] perf tools: Try to show pretty printed output for dynamic sort keys
Em Tue, Dec 15, 2015 at 10:03:29AM +0100, Jiri Olsa escreveu:
Quoted 28 lines · Namhyung Kim
Quoted 18 lines
Each tracepoint event has format string for print to improve readability. Try to parse the output and match the field name. If it finds one, use that for the result. If not, fallbacks to the original output.
For example, sort on kmem:kmalloc.gfp_flags looks like below:
(Note: libtraceevent plugins are not installed on my system. They might affect the output below)Before:
# Overhead Commandgfp_flags# ........ ....... .......... # 99.89% perf 32848 0.06% sleep 208 0.03% perf 32976 0.01% perf 208Quoted 8 lines
After:
# Overhead Commandgfp_flags# ........ ....... ................... # 99.89% perf GFP_NOFS|GFP_ZERO 0.06% sleep GFP_KERNEL 0.03% perf GFP_KERNEL|GFP_ZERO 0.01% perf GFP_KERNEL
Quoted 1 line
hum, maybe we want some way to switch back to numbers?
Or remove repetitive stuff like GFP_? I guess this was done already for perf kmem?
- Arnaldo
Namhyung Kim
maintainerRe: [PATCH 6/6] perf tools: Try to show pretty printed output for dynamic sort keys
Quoted 30 lines · Arnaldo Carvalho de Melo
Em Tue, Dec 15, 2015 at 10:03:29AM +0100, Jiri Olsa escreveu:
Quoted 29 lines · Namhyung Kim
Quoted 18 lines
Each tracepoint event has format string for print to improve readability. Try to parse the output and match the field name. If it finds one, use that for the result. If not, fallbacks to the original output.
For example, sort on kmem:kmalloc.gfp_flags looks like below:
(Note: libtraceevent plugins are not installed on my system. They might affect the output below)Before:
# Overhead Commandgfp_flags# ........ ....... .......... # 99.89% perf 32848 0.06% sleep 208 0.03% perf 32976 0.01% perf 208Quoted 8 lines
After:
# Overhead Commandgfp_flags# ........ ....... ................... # 99.89% perf GFP_NOFS|GFP_ZERO 0.06% sleep GFP_KERNEL 0.03% perf GFP_KERNEL|GFP_ZERO 0.01% perf GFP_KERNELhum, maybe we want some way to switch back to numbers?
OK. Maybe something like 'kmem:kmalloc.gfp_flags/raw' ?
Quoted 3 lines
Or remove repetitive stuff like
GFP_? I guess this was done already for perf kmem?
Right, perf-kmem did that. But perf-report is more general and it's hard to handle every field this way. Also I think it might be better to keep the output as perf-script.
Thanks,
Namhyung
Re: [PATCH 6/6] perf tools: Try to show pretty printed output for dynamic sort keys
Quoted 30 lines · Namhyung Kim
Quoted 29 lines · Arnaldo Carvalho de Melo
Em Tue, Dec 15, 2015 at 10:03:29AM +0100, Jiri Olsa escreveu:
Quoted 28 lines · Namhyung Kim
Quoted 26 lines
Each tracepoint event has format string for print to improve readability. Try to parse the output and match the field name. If it finds one, use that for the result. If not, fallbacks to the original output.
For example, sort on kmem:kmalloc.gfp_flags looks like below:
(Note: libtraceevent plugins are not installed on my system. They might affect the output below)Before:
# Overhead Commandgfp_flags# ........ ....... .......... # 99.89% perf 32848 0.06% sleep 208 0.03% perf 32976 0.01% perf 208After:
# Overhead Commandgfp_flags# ........ ....... ................... # 99.89% perf GFP_NOFS|GFP_ZERO 0.06% sleep GFP_KERNEL 0.03% perf GFP_KERNEL|GFP_ZERO 0.01% perf GFP_KERNEL
Quoted 3 lines
Quoted 1 line
Quoted 1 line
hum, maybe we want some way to switch back to numbers?
OK. Maybe something like 'kmem:kmalloc.gfp_flags/raw' ?
ok.. and some option that would make it happen globaly ;-)
jirka
Namhyung Kim
maintainerRe: [PATCH 6/6] perf tools: Try to show pretty printed output for dynamic sort keys
Quoted 31 lines · Jiri Olsa
Quoted 30 lines · Namhyung Kim
Quoted 29 lines · Arnaldo Carvalho de Melo
Em Tue, Dec 15, 2015 at 10:03:29AM +0100, Jiri Olsa escreveu:
Quoted 28 lines · Namhyung Kim
Quoted 26 lines
Each tracepoint event has format string for print to improve readability. Try to parse the output and match the field name. If it finds one, use that for the result. If not, fallbacks to the original output.
For example, sort on kmem:kmalloc.gfp_flags looks like below:
(Note: libtraceevent plugins are not installed on my system. They might affect the output below)Before:
# Overhead Commandgfp_flags# ........ ....... .......... # 99.89% perf 32848 0.06% sleep 208 0.03% perf 32976 0.01% perf 208After:
# Overhead Commandgfp_flags# ........ ....... ................... # 99.89% perf GFP_NOFS|GFP_ZERO 0.06% sleep GFP_KERNEL 0.03% perf GFP_KERNEL|GFP_ZERO 0.01% perf GFP_KERNEL
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
hum, maybe we want some way to switch back to numbers?
OK. Maybe something like 'kmem:kmalloc.gfp_flags/raw' ?
ok.. and some option that would make it happen globaly ;-)
How about '--raw-trace' then?
Thanks,
Namhyung
Re: [PATCH 6/6] perf tools: Try to show pretty printed output for dynamic sort keys
Quoted 32 lines · Namhyung Kim
Quoted 31 lines · Jiri Olsa
Quoted 30 lines · Namhyung Kim
Quoted 29 lines · Arnaldo Carvalho de Melo
Em Tue, Dec 15, 2015 at 10:03:29AM +0100, Jiri Olsa escreveu:
Quoted 28 lines · Namhyung Kim
Quoted 26 lines
Each tracepoint event has format string for print to improve readability. Try to parse the output and match the field name. If it finds one, use that for the result. If not, fallbacks to the original output.
For example, sort on kmem:kmalloc.gfp_flags looks like below:
(Note: libtraceevent plugins are not installed on my system. They might affect the output below)Before:
# Overhead Commandgfp_flags# ........ ....... .......... # 99.89% perf 32848 0.06% sleep 208 0.03% perf 32976 0.01% perf 208After:
# Overhead Commandgfp_flags# ........ ....... ................... # 99.89% perf GFP_NOFS|GFP_ZERO 0.06% sleep GFP_KERNEL 0.03% perf GFP_KERNEL|GFP_ZERO 0.01% perf GFP_KERNEL
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
hum, maybe we want some way to switch back to numbers?
OK. Maybe something like 'kmem:kmalloc.gfp_flags/raw' ?
ok.. and some option that would make it happen globaly ;-)
How about '--raw-trace' then?
sounds good to me
thanks,
jirka
Arnaldo Carvalho de Melo
maintainerEm Tue, Dec 15, 2015 at 12:46:07AM +0900, Namhyung Kim escreveu:
Quoted 97 lines
Hello,
This is an attempt to improve perf to deal with tracepoint events better. The perf tools can handle tracepoint events but perf report on them is less useful since they're always sampled in a fixed location and not provide event specific info. We can use perf script but I always wishes there's more convenient way to see the result.
I suggest dynamic sort keys created for each event.field on demand. Consider following example:
# Overhead Command Shared Object Symbol # ........ ............... ................ .............. # 47.22% swapper [kernel.vmlinux] [k] __schedule 21.67% transmission-gt [kernel.vmlinux] [k] __schedule 8.23% netctl-auto [kernel.vmlinux] [k] __schedule 5.53% kworker/0:1H [kernel.vmlinux] [k] __schedule 1.98% Xephyr [kernel.vmlinux] [k] __schedule 1.33% irq/33-iwlwifi [kernel.vmlinux] [k] __schedule 1.17% wpa_cli [kernel.vmlinux] [k] __schedule 1.13% rcu_preempt [kernel.vmlinux] [k] __schedule 0.85% ksoftirqd/0 [kernel.vmlinux] [k] __schedule 0.77% Timer [kernel.vmlinux] [k] __schedule...
Currently perf report only shows this but important info is on the event fields, that is:
# sudo cat /sys/kernel/debug/tracing/events/sched/sched_switch/format name: sched_switch ID: 268 format: field:unsigned short common_type; offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2; size:1; signed:0; field:unsigned char common_preempt_count; offset:3; size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1;field:char prev_comm[16]; offset:8; size:16; signed:1; field:pid_t prev_pid; offset:24; size:4; signed:1; field:int prev_prio; offset:28; size:4; signed:1; field:long prev_state; offset:32; size:8; signed:1; field:char next_comm[16]; offset:40; size:16; signed:1; field:pid_t next_pid; offset:56; size:4; signed:1; field:int next_prio; offset:60; size:4; signed:1;print fmt: "prev_comm=%s prev_pid=%d prev_prio=%d prev_state=%s%s ==>
next_comm=%s next_pid=%d next_prio=%d", REC->prev_comm, REC->prev_pid, REC->prev_prio, REC->prev_state & (2048-1) ? __print_flags(REC->prev_state & (2048-1), "|", { 1, "S"} , { 2, "D" }, { 4, "T" }, { 8, "t" }, { 16, "Z" }, { 32, "X" }, { 64, "x" }, { 128, "K"}, { 256, "W" }, { 512, "P" }, { 1024, "N" }) : "R", REC->prev_state & 2048 ? "+" : "", REC->next_comm, REC->next_pid, REC->next_prioWith dynamic sort keys, you can use <event.field> as a sort key. Those dynamic keys are checked and created on demand. For instance, below is to sort by
next_pidfield on the same data file.$ perf report -s comm,sched:sched_switch.next_pid --stdio ... # Overhead Command next_pid # ........ ............... .......... # 21.23% transmission-gt 0 20.86% swapper 17773 6.62% netctl-auto 0 5.25% swapper 109 5.21% kworker/0:1H 0 1.98% Xephyr 0 1.98% swapper 6524 1.98% swapper 27478 1.37% swapper 27476 1.17% swapper 233Multiple dynamic sort keys are also supported:
$ perf report -s comm,sched:sched_switch.next_pid,sched:sched_switch.next_comm --stdio ... # Overhead Command next_pid next_comm # ........ ............... .......... ................ # 20.86% swapper 17773 transmission-gt 9.64% transmission-gt 0 swapper/0 9.16% transmission-gt 0 swapper/2 5.25% swapper 109 kworker/0:1H 5.21% kworker/0:1H 0 swapper/0 2.14% netctl-auto 0 swapper/2 1.98% netctl-auto 0 swapper/0 1.98% swapper 6524 Xephyr 1.98% swapper 27478 netctl-auto 1.78% transmission-gt 0 swapper/3 1.53% Xephyr 0 swapper/0 1.29% netctl-auto 0 swapper/1 1.29% swapper 27476 netctl-auto 1.21% netctl-auto 0 swapper/3 1.17% swapper 233 irq/33-iwlwifi
Quoted 8 lines
Note that pid 0 exists for each cpu so have comm of 'swapper/N'.
This is available on 'perf/dynamic-sort-v1' branch in my tree
git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.gitAny comments are welcome, thanks!
Namhyung
I'll look at the patches for style, but the idea is so nice and natural I thought about blind merging it :-)
- Arnaldo
Quoted 47 lines · Arnaldo Carvalho de Melo
Quoted 47 lines
With dynamic sort keys, you can use <event.field> as a sort key. Those dynamic keys are checked and created on demand. For instance, below is to sort by
next_pidfield on the same data file.$ perf report -s comm,sched:sched_switch.next_pid --stdio ... # Overhead Command next_pid # ........ ............... .......... # 21.23% transmission-gt 0 20.86% swapper 17773 6.62% netctl-auto 0 5.25% swapper 109 5.21% kworker/0:1H 0 1.98% Xephyr 0 1.98% swapper 6524 1.98% swapper 27478 1.37% swapper 27476 1.17% swapper 233Multiple dynamic sort keys are also supported:
$ perf report -s comm,sched:sched_switch.next_pid,sched:sched_switch.next_comm --stdio ... # Overhead Command next_pid next_comm # ........ ............... .......... ................ # 20.86% swapper 17773 transmission-gt 9.64% transmission-gt 0 swapper/0 9.16% transmission-gt 0 swapper/2 5.25% swapper 109 kworker/0:1H 5.21% kworker/0:1H 0 swapper/0 2.14% netctl-auto 0 swapper/2 1.98% netctl-auto 0 swapper/0 1.98% swapper 6524 Xephyr 1.98% swapper 27478 netctl-auto 1.78% transmission-gt 0 swapper/3 1.53% Xephyr 0 swapper/0 1.29% netctl-auto 0 swapper/1 1.29% swapper 27476 netctl-auto 1.21% netctl-auto 0 swapper/3 1.17% swapper 233 irq/33-iwlwifiNote that pid 0 exists for each cpu so have comm of 'swapper/N'.
This is available on 'perf/dynamic-sort-v1' branch in my tree
Quoted 8 lines
Quoted 4 lines
git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.gitAny comments are welcome, thanks!
NamhyungI'll look at the patches for style, but the idea is so nice and natural I thought about blind merging it :-)
yes, that is a cool feature.
For scheduling tracepoints the analysis could be added to perf-sched to ease the burden of the command line syntax.
Namhyung Kim
maintainerHi David and Arnaldo,
Quoted 48 lines · David Ahern
Quoted 47 lines · Arnaldo Carvalho de Melo
Quoted 47 lines
With dynamic sort keys, you can use <event.field> as a sort key. Those dynamic keys are checked and created on demand. For instance, below is to sort by
next_pidfield on the same data file.$ perf report -s comm,sched:sched_switch.next_pid --stdio ... # Overhead Command next_pid # ........ ............... .......... # 21.23% transmission-gt 0 20.86% swapper 17773 6.62% netctl-auto 0 5.25% swapper 109 5.21% kworker/0:1H 0 1.98% Xephyr 0 1.98% swapper 6524 1.98% swapper 27478 1.37% swapper 27476 1.17% swapper 233Multiple dynamic sort keys are also supported:
$ perf report -s comm,sched:sched_switch.next_pid,sched:sched_switch.next_comm --stdio ... # Overhead Command next_pid next_comm # ........ ............... .......... ................ # 20.86% swapper 17773 transmission-gt 9.64% transmission-gt 0 swapper/0 9.16% transmission-gt 0 swapper/2 5.25% swapper 109 kworker/0:1H 5.21% kworker/0:1H 0 swapper/0 2.14% netctl-auto 0 swapper/2 1.98% netctl-auto 0 swapper/0 1.98% swapper 6524 Xephyr 1.98% swapper 27478 netctl-auto 1.78% transmission-gt 0 swapper/3 1.53% Xephyr 0 swapper/0 1.29% netctl-auto 0 swapper/1 1.29% swapper 27476 netctl-auto 1.21% netctl-auto 0 swapper/3 1.17% swapper 233 irq/33-iwlwifiNote that pid 0 exists for each cpu so have comm of 'swapper/N'.
This is available on 'perf/dynamic-sort-v1' branch in my tree
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.gitAny comments are welcome, thanks!
NamhyungI'll look at the patches for style, but the idea is so nice and natural I thought about blind merging it :-)
Hehe, thanks!
Quoted 5 lines
yes, that is a cool feature.
For scheduling tracepoints the analysis could be added to perf-sched to ease the burden of the command line syntax.
Yes, the existing tracepoints analysis command might use or wrap it.
For command line syntax, I think it'd be better to provide shorter way to identify the events - like (non-ambiguous) substring match or positional parameter (e.g. %1 for first event).
For example, the last example above can be:
$ perf report -s comm,switch.next_pid,switch.next_comm
or
$ perf report -s comm,%1.next_pid,%1.next_comm
Thanks,
Namhyung