perf/core improvements and fixes
Jiri Olsa
reviewerhi Ingo,
please consider pulling
thanks,
jirka
The following changes since commit 82b897782d10fcc4930c9d4a15b175348fdd2871:
perf: Differentiate exec() and non-exec() comm events (2014-06-06 07:56:22 +0200)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git tags/perf-core-for-mingo
for you to fetch changes up to a2609f3b0c582d6aaa8f69a61a0eea6c7a98d291:
perf tools: Support spark lines in perf stat (2014-06-09 13:34:50 +0200)
----------------------------------------------------------------
perf/core improvements and fixes:
. Bitmask handling and plugin updates (Steven Rostedt)
. Fix pipe check regression in attr event callback (Jiri Olsa)
. Prettify the tags/TAGS/cscope targets output (Jiri Olsa)
. Print array argument as string (Namhyung Kim)
. Pass protection and flags bits through mmap2 interface (Peter Zijlstra)
. Update perf tool mmap2 interface with protection and flag bits (Don Zickus)
. Re-enable mmap interface (Don Zickus)
. Add mem-mode documentation to report command (Don Zickus)
. Add sort on dcacheline (Don Zickus)
. Support spark lines in perf stat (Andi Kleen)
----------------------------------------------------------------
Andi Kleen (1):
perf tools: Support spark lines in perf stat
Don Zickus (6):
perf tools: Update mmap2 interface with protection and flag bits Revert "perf: Disable PERF_RECORD_MMAP2 support"
perf report: Add mem-mode documentation to report command perf tools: Add cpumode to struct hist_entry
perf tools: Add support to dynamically get cacheline size perf tools: Add dcacheline sort
Jiri Olsa (2):
perf tools: Fix pipe check regression in attr event callback perf tools: Prettify the tags/TAGS/cscope targets output
Namhyung Kim (1):
perf script/python: Print array argument as string
Peter Zijlstra (1):
perf: Pass protection and flags bits through mmap2 interface
Steven Rostedt (1):
tools lib traceevent: Add options to plugins
Steven Rostedt (Red Hat) (3):
tools lib traceevent: Add flag to not load event plugins tools lib traceevent: Add options to function plugin tools lib traceevent: Added support for __get_bitmask() macro
include/uapi/linux/perf_event.h | 1 + kernel/events/core.c | 37 +++- tools/lib/traceevent/event-parse.c | 113 ++++++++++++ tools/lib/traceevent/event-parse.h | 25 ++- tools/lib/traceevent/event-plugin.c | 203 ++++++++++++++++++++- tools/lib/traceevent/plugin_function.c | 43 ++++- tools/perf/Documentation/perf-report.txt | 23 +++ tools/perf/Documentation/perf-stat.txt | 4 + tools/perf/Makefile.perf | 7 +- tools/perf/builtin-inject.c | 2 +- tools/perf/builtin-stat.c | 12 ++ tools/perf/perf.c | 1 + tools/perf/tests/dwarf-unwind.c | 2 +- tools/perf/util/event.c | 57 ++++-- tools/perf/util/event.h | 2 + tools/perf/util/evsel.c | 1 + tools/perf/util/hist.c | 9 +- tools/perf/util/hist.h | 1 + tools/perf/util/machine.c | 4 +- tools/perf/util/map.c | 4 +- tools/perf/util/map.h | 4 +- .../perf/util/scripting-engines/trace-event-perl.c | 1 + .../util/scripting-engines/trace-event-python.c | 2 + tools/perf/util/sort.c | 107 +++++++++++ tools/perf/util/sort.h | 2 + tools/perf/util/spark.c | 31 ++++ tools/perf/util/spark.h | 4 + tools/perf/util/stat.c | 34 ++++ tools/perf/util/stat.h | 10 + tools/perf/util/util.c | 1 + tools/perf/util/util.h | 1 + 31 files changed, 707 insertions(+), 41 deletions(-) create mode 100644 tools/perf/util/spark.c create mode 100644 tools/perf/util/spark.h
Jiri Olsa
reviewerPATCH 1/15 tools lib traceevent: Add flag to not load event plugins
From: "Steven Rostedt (Red Hat)" <[email protected]>
Add a flag to pevent that will let the callers be able to set it and keep the system, and perhaps even normal plugins from being loaded.
This is useful when plugins might hide certain information and seeing the raw events shows what may be going on.
--- tools/lib/traceevent/event-parse.h | 2 ++ tools/lib/traceevent/event-plugin.c | 7 ++++++- 2 files changed, 8 insertions(+), 1 deletion(-)
tools/lib/traceevent/event-parse.h +2 -0
@@ -354,6 +354,8 @@ enum pevent_func_arg_type {enum pevent_flag {PEVENT_NSEC_OUTPUT =1 ,/* output in NSECS */ +PEVENT_DISABLE_SYS_PLUGINS =1 <<1 , +PEVENT_DISABLE_PLUGINS =1 <<2 , };#define PEVENT_ERRORS \
tools/lib/traceevent/event-plugin.c +6 -1
@@ -148,12 +148,17 @@ load_plugins(struct pevent *pevent, const char *suffix,char *path;char *envdir; +if (pevent->flags &PEVENT_DISABLE_PLUGINS ) +return ; +/* * If a system plugin directory was defined, * check that first. */ #ifdef PLUGIN_DIR -load_plugins_dir (pevent, suffix,PLUGIN_DIR , load_plugin, data); +if (!(pevent->flags &PEVENT_DISABLE_SYS_PLUGINS )) +load_plugins_dir (pevent, suffix,PLUGIN_DIR , + load_plugin, data);#endif /*
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 2/15 tools lib traceevent: Add options to plugins
In reply to Jiri OlsaFrom: Steven Rostedt <[email protected]>
The traceevent plugins allows developers to have their events print out information that is more advanced than what can be achieved by the trace event format files.
As these plugins are used on the userspace side of the tracing tools, it is only logical that the tools should be able to produce different types of output for the events. The types of events still need to be defined by the plugins thus we need a way to pass information from the tool to the plugin to specify what type of information to be shown.
Not only does the information need to be passed by the tool to plugin, but the plugin also requires a way to notify the tool of what options it can provide.
This builds the plugin option infrastructure that is taken from trace-cmd that is used to allow plugins to produce different output based on the options specified by the tool.
--- tools/lib/traceevent/event-parse.h | 16 ++- tools/lib/traceevent/event-plugin.c | 196 ++++++++++++++++++++++++++++++++++++ 2 files changed, 209 insertions(+), 3 deletions(-)
tools/lib/traceevent/event-parse.h +13 -3
@@ -107,8 +107,8 @@ typedef int (*pevent_event_handler_func)(struct trace_seq *s,typedef int (*pevent_plugin_load_func )(struct pevent *pevent);typedef int (*pevent_plugin_unload_func )(struct pevent *pevent); -struct plugin_option { -struct plugin_option *next; +struct pevent_plugin_option { +struct pevent_plugin_option *next;void *handle;char *file;char *name;
@@ -135,7 +135,7 @@ struct plugin_option {* PEVENT_PLUGIN_OPTIONS: (optional) * Plugin options that can be set before loading * -* struct plugin_option PEVENT_PLUGIN_OPTIONS[] = { +* struct pevent_plugin_option PEVENT_PLUGIN_OPTIONS[] = { * { * .name = "option-name", * .plugin_alias = "overide-file-name", (optional)
@@ -412,9 +412,19 @@ enum pevent_errno {struct plugin_list ; +#define INVALID_PLUGIN_LIST_OPTION ((char **)((unsigned long)-1)) +struct plugin_list *traceevent_load_plugins (struct pevent *pevent);void traceevent_unload_plugins (struct plugin_list *plugin_list,struct pevent *pevent); +char **traceevent_plugin_list_options (void ); +void traceevent_plugin_free_options_list (char **list); +int traceevent_plugin_add_options (const char *name, +struct pevent_plugin_option *options); +void traceevent_plugin_remove_options (struct pevent_plugin_option *options); +void traceevent_print_plugins (struct trace_seq *s, +const char *prefix,const char *suffix, +const struct plugin_list *list);struct cmdline ;struct cmdline_list ;
tools/lib/traceevent/event-plugin.c +196 -0
214 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 3/15 tools lib traceevent: Add options to function plugin
In reply to Jiri OlsaFrom: "Steven Rostedt (Red Hat)" <[email protected]>
Add the options "parent" and "indent" to the function plugin.
When parent is set, the output looks like this:
function: fsnotify_modify <-- vfs_write function: zone_statistics <-- get_page_from_freelist function: __inc_zone_state <-- zone_statistics function: inotify_inode_queue_event <-- fsnotify_modify function: fsnotify_parent <-- fsnotify_modify function: __inc_zone_state <-- zone_statistics function: __fsnotify_parent <-- fsnotify_parent function: inotify_dentry_parent_queue_event <-- fsnotify_parent function: add_to_page_cache_lru <-- do_read_cache_page
When it's not set, it looks like:
function: fsnotify_modify function: zone_statistics function: __inc_zone_state function: inotify_inode_queue_event function: fsnotify_parent function: __inc_zone_state function: __fsnotify_parent function: inotify_dentry_parent_queue_event function: add_to_page_cache_lru
When the otpion "indent" is not set, it looks like this:
function: fsnotify_modify <-- vfs_write function: zone_statistics <-- get_page_from_freelist function: __inc_zone_state <-- zone_statistics function: inotify_inode_queue_event <-- fsnotify_modify function: fsnotify_parent <-- fsnotify_modify function: __inc_zone_state <-- zone_statistics function: __fsnotify_parent <-- fsnotify_parent function: inotify_dentry_parent_queue_event <-- fsnotify_parent function: add_to_page_cache_lru <-- do_read_cache_page
--- tools/lib/traceevent/plugin_function.c | 43 +++++++++++++++++++++++++++++----- 1 file changed, 37 insertions(+), 6 deletions(-)
tools/lib/traceevent/plugin_function.c +37 -6
@@ -33,6 +33,29 @@ static int cpus = -1;#define STK_BLK 10 +struct pevent_plugin_option plugin_options[] = +{ + { + .name ="parent" , + .plugin_alias ="ftrace" , + .description = +"Print parent of functions for function events" , + }, + { + .name ="indent" , + .plugin_alias ="ftrace" , + .description = +"Try to show function call indents, based on parents" , + .set =1 , + }, + { + .name =NULL , + } +}; + +static struct pevent_plugin_option *ftrace_parent = &plugin_options[0 ]; +static struct pevent_plugin_option *ftrace_indent = &plugin_options[1 ]; +static void add_child (struct func_stack *stack,const char *child,int pos) {int i;
@@ -119,7 +142,8 @@ static int function_handler(struct trace_seq *s, struct pevent_record *record, parent =pevent_find_function (pevent, pfunction); - index =add_and_get_index (parent, func, record->cpu); +if (parent && ftrace_indent->set) + index =add_and_get_index (parent, func, record->cpu);trace_seq_printf (s,"%*s" , index*3 ,"" );
@@ -128,11 +152,13 @@ static int function_handler(struct trace_seq *s, struct pevent_record *record,else trace_seq_printf (s,"0x%llx" , function); -trace_seq_printf (s," <-- " ); -if (parent) -trace_seq_printf (s,"%s" , parent); -else -trace_seq_printf (s,"0x%llx" , pfunction); +if (ftrace_parent->set) { +trace_seq_printf (s," <-- " ); +if (parent) +trace_seq_printf (s,"%s" , parent); +else +trace_seq_printf (s,"0x%llx" , pfunction); + }return 0 ; }
@@ -141,6 +167,9 @@ int PEVENT_PLUGIN_LOADER(struct pevent *pevent) {pevent_register_event_handler (pevent,-1 ,"ftrace" ,"function" , function_handler,NULL ); + +traceevent_plugin_add_options ("ftrace" , plugin_options); +return 0 ; }
@@ -157,6 +186,8 @@ void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)free (fstack[i].stack); } +traceevent_plugin_remove_options (plugin_options); +free (fstack); fstack =NULL ; cpus =-1 ;
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 4/15 tools lib traceevent: Added support for __get_bitmask() macro
In reply to Jiri OlsaFrom: "Steven Rostedt (Red Hat)" <[email protected]>
Coming in v3.16, trace events will be able to save bitmasks in raw format in the ring buffer and output it with the __get_bitmask() macro.
In order for userspace tools to parse this, it must be able to handle the __get_bitmask() call and be able to convert the data that's in the ring buffer into a nice bitmask format. The output is similar to what the kernel uses to print bitmasks, with a comma separator every 4 bytes (8 characters).
This allows for cpumasks to also be saved efficiently.
The first user is the thermal:thermal_power_limit event which has the following output:
thermal_power_limit: cpus=0000000f freq=1900000 cdev_state=0 power=5252
--- tools/lib/traceevent/event-parse.c | 113 +++++++++++++++++++++ tools/lib/traceevent/event-parse.h | 7 ++ .../perf/util/scripting-engines/trace-event-perl.c | 1 + .../util/scripting-engines/trace-event-python.c | 1 + 4 files changed, 122 insertions(+)
tools/lib/traceevent/event-parse.c +113 -0
@@ -765,6 +765,9 @@ static void free_arg(struct print_arg *arg)case PRINT_BSTRING :free (arg->string.string);break ; +case PRINT_BITMASK : +free (arg->bitmask.bitmask); +break ;case PRINT_DYNAMIC_ARRAY :free (arg->dynarray.index);break ;
@@ -2268,6 +2271,7 @@ static int arg_num_eval(struct print_arg *arg, long long *val) case PRINT_FIELD ... PRINT_SYMBOL:case PRINT_STRING :case PRINT_BSTRING : +case PRINT_BITMASK :default :do_warning ("invalid eval type %d" , arg->type); ret =0 ;
@@ -2296,6 +2300,7 @@ static char *arg_eval (struct print_arg *arg) case PRINT_FIELD ... PRINT_SYMBOL:case PRINT_STRING :case PRINT_BSTRING : +case PRINT_BITMASK :default :do_warning ("invalid eval type %d" , arg->type);break ;
@@ -2683,6 +2688,35 @@ process_str(struct event_format *event __maybe_unused, struct print_arg *arg,return EVENT_ERROR ; } +static enum event_type +process_bitmask (struct event_format *event__maybe_unused ,struct print_arg *arg, +char **tok) +{ +enum event_type type; +char *token; + +if (read_expect_type (EVENT_ITEM , &token) <0 ) + goto out_free; + + arg->type =PRINT_BITMASK ; + arg->bitmask.bitmask = token; + arg->bitmask.offset =-1 ; + +if (read_expected (EVENT_DELIM ,")" ) <0 ) + goto out_err; + + type =read_token (&token); + *tok = token; + +return type; + + out_free: +free_token (token); + out_err: + *tok =NULL ; +return EVENT_ERROR ; +} +static struct pevent_function_handler *find_func_handler (struct pevent *pevent,char *func_name) {
@@ -2797,6 +2831,10 @@ process_function(struct event_format *event, struct print_arg *arg,free_token (token);return process_str (event, arg, tok); } +if (strcmp (token,"__get_bitmask" ) ==0 ) { +free_token (token); +return process_bitmask (event, arg, tok); + }if (strcmp (token,"__get_dynamic_array" ) ==0 ) {free_token (token);return process_dynamic_array (event, arg, tok);
@@ -3324,6 +3362,7 @@ eval_num_arg(void *data, int size, struct event_format *event, struct print_argreturn eval_type (val, arg,0 );case PRINT_STRING :case PRINT_BSTRING : +case PRINT_BITMASK :return 0 ;case PRINT_FUNC : {struct trace_seq s;
@@ -3556,6 +3595,60 @@ static void print_str_to_seq(struct trace_seq *s, const char *format,trace_seq_printf (s, format, str); } +static void print_bitmask_to_seq (struct pevent *pevent, +struct trace_seq *s,const char *format, +int len_arg,const void *data,int size) +{ +int nr_bits = size *8 ; +int str_size = (nr_bits +3 ) /4 ; +int len =0 ; +char buf[3 ]; +char *str; +int index; +int i; + +/* +* The kernel likes to put in commas every 32 bits, we +* can do the same. +*/ + str_size += (nr_bits -1 ) /32 ; + + str =malloc (str_size +1 ); +if (!str) { +do_warning ("%s: not enough memory!" , __func__); +return ; + } + str[str_size] =0 ; + +/* Start out with -2 for the two chars per byte */ +for (i = str_size -2 ; i >=0 ; i -=2 ) { +/* +* data points to a bit mask of size bytes. +* In the kernel, this is an array of long words, thus +* endianess is very important. +*/ +if (pevent->file_bigendian) + index = size - (len +1 ); +else + index = len; + +snprintf (buf,3 ,"%02x" , *((unsigned char *)data + index)); +memcpy (str + i, buf,2 ); + len++; +if (!(len &3 ) && i >0 ) { + i--; + str[i] =',' ; + } + } + +if (len_arg >=0 ) +trace_seq_printf (s, format, len_arg, str); +else +trace_seq_printf (s, format, str); + +free (str); +} +static void print_str_arg (struct trace_seq *s,void *data,int size,struct event_format *event,const char *format,int len_arg,struct print_arg *arg)
@@ -3691,6 +3784,23 @@ static void print_str_arg(struct trace_seq *s, void *data, int size,case PRINT_BSTRING :print_str_to_seq (s, format, len_arg, arg->string.string);break ; +case PRINT_BITMASK : { +int bitmask_offset; +int bitmask_size; + +if (arg->bitmask.offset ==-1 ) { +struct format_field *f; + + f =pevent_find_any_field (event, arg->bitmask.bitmask); + arg->bitmask.offset = f->offset; + } + bitmask_offset =data2host4 (pevent, data + arg->bitmask.offset); + bitmask_size = bitmask_offset >>16 ; + bitmask_offset &=0xffff ; +print_bitmask_to_seq (pevent, s, format, len_arg, + data + bitmask_offset, bitmask_size); +break ; + }case PRINT_OP :/* * The only op for string should be ? :
@@ -4822,6 +4932,9 @@ static void print_args(struct print_arg *args)case PRINT_BSTRING :printf ("__get_str(%s)" , args->string.string);break ; +case PRINT_BITMASK : +printf ("__get_bitmask(%s)" , args->bitmask.bitmask); +break ;case PRINT_TYPE :printf ("(%s)" , args->typecast.type);print_args (args->typecast.item);
tools/lib/traceevent/event-parse.h +7 -0
@@ -208,6 +208,11 @@ struct print_arg_string {int offset; }; +struct print_arg_bitmask { +char *bitmask; +int offset; +}; +struct print_arg_field {char *name;struct format_field *field;
@@ -274,6 +279,7 @@ enum print_arg_type {PRINT_DYNAMIC_ARRAY ,PRINT_OP ,PRINT_FUNC , + PRINT_BITMASK, }; struct print_arg {
@@ -288,6 +294,7 @@ struct print_arg {struct print_arg_hex hex;struct print_arg_func func;struct print_arg_string string; +struct print_arg_bitmask bitmask;struct print_arg_op op;struct print_arg_dynarray dynarray; };
tools/perf/util/scripting-engines/trace-event-perl.c +1 -0
@@ -215,6 +215,7 @@ static void define_event_symbols(struct event_format *event,case PRINT_BSTRING :case PRINT_DYNAMIC_ARRAY :case PRINT_STRING : +case PRINT_BITMASK :break ;case PRINT_TYPE :define_event_symbols (event, ev_name, args->typecast.item);
tools/perf/util/scripting-engines/trace-event-python.c +1 -0
@@ -197,6 +197,7 @@ static void define_event_symbols(struct event_format *event,case PRINT_BSTRING :case PRINT_DYNAMIC_ARRAY :case PRINT_FUNC : +case PRINT_BITMASK :/* we should warn... */ return ; }
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 5/15 perf tools: Fix pipe check regression in attr event callback
In reply to Jiri OlsaThe file factoring in builtin-inject.c object introduced regression in attr event callback. The commit is:
3406912 perf inject: Handle output file via perf_data_file object
Following hunk reversed the logic:
- if (!inject->pipe_output)
+ if (&inject->output.is_pipe)
putting it back, following example now works:
$ perf record -o - kill | perf inject -b | perf report -i -
Plus removing extra '&' (kudos to Arnaldo)
--- tools/perf/builtin-inject.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/builtin-inject.c +1 -1
@@ -72,7 +72,7 @@ static int perf_event__repipe_attr(struct perf_tool *tool,if (ret)return ret; -if (&inject->output.is_pipe) +if (!inject->output.is_pipe)return 0 ;return perf_event__repipe_synth (tool, event);
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 6/15 perf tools: Prettify the tags/TAGS/cscope targets output
In reply to Jiri OlsaAdd tags/TAGS/cscope targets to the quiet family.
$ make tags cscope BUILD: Doing 'make -j4' parallel build GEN tags
$ make cscope BUILD: Doing 'make -j4' parallel build GEN cscope
--- tools/perf/Makefile.perf | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/perf/Makefile.perf +3 -3
@@ -819,15 +819,15 @@ TAG_FOLDERS= . ../lib/traceevent ../lib/api ../lib/symbol TAG_FILES= ../../include/uapi/linux/perf_event.h TAGS: - $(RM) TAGS + $(QUIET_GEN)$(RM) TAGS; \ $(FIND) $(TAG_FOLDERS) -name '*.[hcS]' -print | xargs etags -a $(TAG_FILES) tags: - $(RM) tags + $(QUIET_GEN)$(RM) tags; \ $(FIND) $(TAG_FOLDERS) -name '*.[hcS]' -print | xargs ctags -a $(TAG_FILES) cscope: - $(RM) cscope* + $(QUIET_GEN)$(RM) cscope*; \ $(FIND) $(TAG_FOLDERS) -name '*.[hcS]' -print | xargs cscope -b $(TAG_FILES) ### Detect prefix changes
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 7/15 perf script/python: Print array argument as string
In reply to Jiri OlsaFrom: Namhyung Kim <[email protected]>
With the Sebastian's change of handling num array argument (of raw syscall enter), the script still failed to work like this:
$ perf record -e raw_syscalls:* sleep 1
$ perf script -g python
$ perf script -s perf-script.py
...
Traceback (most recent call last):
File "perf-script.py", line 42, in raw_syscalls__sys_enter
(id, args),
TypeError: %u format: a number is required, not list
Fatal Python error: problem in Python trace event handler
Aborted (core dumped)This is because the generated script tries to print the array arg as unsigned integer (%u). Since the python seems to convert arguments to strings by default, just using %s solved the problem for me.
--- tools/perf/util/scripting-engines/trace-event-python.c | 1 + 1 file changed, 1 insertion(+)
tools/perf/util/scripting-engines/trace-event-python.c +1 -0
@@ -623,6 +623,7 @@ static int python_generate_script(struct pevent *pevent, const char *outfile)fprintf (ofp,"%s=" , f->name);if (f->flags &FIELD_IS_STRING || f->flags &FIELD_IS_FLAG || + f->flags &FIELD_IS_ARRAY || f->flags &FIELD_IS_SYMBOLIC )fprintf (ofp,"%%s" );else if (f->flags &FIELD_IS_SIGNED )
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 8/15 perf: Pass protection and flags bits through mmap2 interface
In reply to Jiri OlsaFrom: Peter Zijlstra <[email protected]>
The mmap2 interface was missing the protection and flags bits needed to accurately determine if a mmap memory area was shared or private and if it was readable or not.
--- include/uapi/linux/perf_event.h | 1 + kernel/events/core.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+)
include/uapi/linux/perf_event.h +1 -0
@@ -705,6 +705,7 @@ enum perf_event_type {* u32 min; * u64 ino; * u64 ino_generation; +* u32 prot, flags; * char filename[]; * struct sample_id sample_id; * };
kernel/events/core.c +33 -0
@@ -40,6 +40,7 @@#include <linux/mm_types.h> #include <linux/cgroup.h> #include <linux/module.h> +#include <linux/mman.h> #include "internal.h"
@@ -5127,6 +5128,7 @@ struct perf_mmap_event {int maj, min;u64 ino;u64 ino_generation; +u32 prot, flags;struct {struct perf_event_header header;
@@ -5168,6 +5170,8 @@ static void perf_event_mmap_output(struct perf_event *event, mmap_event->event_id.header.size +=sizeof (mmap_event->min); mmap_event->event_id.header.size +=sizeof (mmap_event->ino); mmap_event->event_id.header.size +=sizeof (mmap_event->ino_generation); + mmap_event->event_id.header.size +=sizeof (mmap_event->prot); + mmap_event->event_id.header.size +=sizeof (mmap_event->flags); }perf_event_header__init_id (&mmap_event->event_id.header, &sample, event);
@@ -5186,6 +5190,8 @@ static void perf_event_mmap_output(struct perf_event *event,perf_output_put (&handle, mmap_event->min);perf_output_put (&handle, mmap_event->ino);perf_output_put (&handle, mmap_event->ino_generation); +perf_output_put (&handle, mmap_event->prot); +perf_output_put (&handle, mmap_event->flags); } __output_copy(&handle, mmap_event->file_name,
@@ -5204,6 +5210,7 @@ static void perf_event_mmap_event(struct perf_mmap_event *mmap_event)struct file *file = vma->vm_file;int maj =0 , min =0 ;u64 ino =0 , gen =0 ; +u32 prot =0 , flags =0 ;unsigned int size;char tmp[16 ];char *buf =NULL ;
@@ -5234,6 +5241,28 @@ static void perf_event_mmap_event(struct perf_mmap_event *mmap_event) gen = inode->i_generation; maj =MAJOR (dev); min =MINOR (dev); + +if (vma->vm_flags &VM_READ ) + prot |=PROT_READ ; +if (vma->vm_flags &VM_WRITE ) + prot |=PROT_WRITE ; +if (vma->vm_flags &VM_EXEC ) + prot |=PROT_EXEC ; + +if (vma->vm_flags &VM_MAYSHARE ) + flags =MAP_SHARED ; +else + flags =MAP_PRIVATE ; + +if (vma->vm_flags &VM_DENYWRITE ) + flags |=MAP_DENYWRITE ; +if (vma->vm_flags &VM_MAYEXEC ) + flags |=MAP_EXECUTABLE ; +if (vma->vm_flags &VM_LOCKED ) + flags |=MAP_LOCKED ; +if (vma->vm_flags &VM_HUGETLB ) + flags |=MAP_HUGETLB ; + goto got_name; }else { name = (char *)arch_vma_name (vma);
@@ -5274,6 +5303,8 @@ got_name: mmap_event->min = min; mmap_event->ino = ino; mmap_event->ino_generation = gen; + mmap_event->prot = prot; + mmap_event->flags = flags;if (!(vma->vm_flags &VM_EXEC )) mmap_event->event_id.header.misc |=PERF_RECORD_MISC_MMAP_DATA ;
@@ -5314,6 +5345,8 @@ void perf_event_mmap(struct vm_area_struct *vma)/* .min (attr_mmap2 only) */ /* .ino (attr_mmap2 only) */ /* .ino_generation (attr_mmap2 only) */ +/* .prot (attr_mmap2 only) */ +/* .flags (attr_mmap2 only) */ };perf_event_mmap_event (&mmap_event);
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 9/15 perf tools: Update mmap2 interface with protection and flag bits
In reply to Jiri OlsaFrom: Don Zickus <[email protected]>
The kernel piece passes more info now. Update the perf tool to reflect that and adjust the synthesized maps to play along.
--- tools/perf/util/event.c | 23 +++++++++++++++++++++-- tools/perf/util/event.h | 2 ++ tools/perf/util/machine.c | 4 +++- tools/perf/util/map.c | 4 +++- tools/perf/util/map.h | 4 +++- 5 files changed, 32 insertions(+), 5 deletions(-)
tools/perf/util/event.c +21 -2
@@ -1,4 +1,5 @@#include <linux/types.h> +#include <sys/mman.h> #include "event.h" #include "debug.h" #include "hist.h"
@@ -212,6 +213,21 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,else event->header.misc =PERF_RECORD_MISC_GUEST_USER ; +/* map protection and flags bits */ + event->mmap2.prot =0 ; + event->mmap2.flags =0 ; +if (prot[0 ] =='r' ) + event->mmap2.prot |=PROT_READ ; +if (prot[1 ] =='w' ) + event->mmap2.prot |=PROT_WRITE ; +if (prot[2 ] =='x' ) + event->mmap2.prot |=PROT_EXEC ; + +if (prot[3 ] =='s' ) + event->mmap2.flags |=MAP_SHARED ; +else + event->mmap2.flags |=MAP_PRIVATE ; +if (prot[2 ] !='x' ) {if (!mmap_data || prot[0 ] !='r' )continue ;
@@ -612,12 +628,15 @@ size_t perf_event__fprintf_mmap(union perf_event *event, FILE *fp)size_t perf_event__fprintf_mmap2 (union perf_event *event,FILE *fp) {return fprintf (fp," %d/%d: [%#" PRIx64"(%#" PRIx64") @ %#" PRIx64 -" %02x:%02x %" PRIu64" %" PRIu64"]: %c %s\n" , +" %02x:%02x %" PRIu64" %" PRIu64"]: %c%c%c%c %s\n" , event->mmap2.pid, event->mmap2.tid, event->mmap2.start, event->mmap2.len, event->mmap2.pgoff, event->mmap2.maj, event->mmap2.min, event->mmap2.ino, event->mmap2.ino_generation, - (event->header.misc &PERF_RECORD_MISC_MMAP_DATA ) ?'r' :'x' , + (event->mmap2.prot &PROT_READ ) ?'r' :'-' , + (event->mmap2.prot &PROT_WRITE ) ?'w' :'-' , + (event->mmap2.prot &PROT_EXEC ) ?'x' :'-' , + (event->mmap2.flags &MAP_SHARED ) ?'s' :'p' , event->mmap2.filename); }
tools/perf/util/event.h +2 -0
@@ -27,6 +27,8 @@ struct mmap2_event {u32 min;u64 ino;u64 ino_generation; +u32 prot; +u32 flags;char filename[PATH_MAX ]; };
tools/perf/util/machine.c +3 -1
@@ -1060,6 +1060,8 @@ int machine__process_mmap2_event(struct machine *machine, event->mmap2.pid, event->mmap2.maj, event->mmap2.min, event->mmap2.ino, event->mmap2.ino_generation, + event->mmap2.prot, + event->mmap2.flags, event->mmap2.filename, type);if (map ==NULL )
@@ -1105,7 +1107,7 @@ int machine__process_mmap_event(struct machine *machine, union perf_event *event map =map__new (&machine->user_dsos, event->mmap.start, event->mmap.len, event->mmap.pgoff, - event->mmap.pid,0 ,0 ,0 ,0 , + event->mmap.pid,0 ,0 ,0 ,0 ,0 ,0 , event->mmap.filename, type);
tools/perf/util/map.c +3 -1
@@ -138,7 +138,7 @@ void map__init(struct map *map, enum map_type type,struct map *map__new (struct list_head *dsos__list,u64 start,u64 len,u64 pgoff,u32 pid,u32 d_maj,u32 d_min,u64 ino, -u64 ino_gen,char *filename, +u64 ino_gen,u32 prot,u32 flags,char *filename,enum map_type type) {struct map *map =malloc (sizeof (*map));
@@ -157,6 +157,8 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len, map->min = d_min; map->ino = ino; map->ino_generation = ino_gen; + map->prot = prot; + map->flags = flags;if ((anon || no_dso) && type ==MAP__FUNCTION ) {snprintf (newfilename,sizeof (newfilename),"/tmp/perf-%d.map" , pid);
tools/perf/util/map.h +3 -1
@@ -35,6 +35,8 @@ struct map {bool referenced;bool erange_warned;u32 priv; +u32 prot; +u32 flags;u64 pgoff;u64 reloc;u32 maj, min;/* only valid for MMAP2 record */
@@ -118,7 +120,7 @@ void map__init(struct map *map, enum map_type type, u64 start, u64 end, u64 pgoff, struct dso *dso);struct map *map__new (struct list_head *dsos__list,u64 start,u64 len,u64 pgoff,u32 pid,u32 d_maj,u32 d_min,u64 ino, -u64 ino_gen, +u64 ino_gen,u32 prot,u32 flags,char *filename,enum map_type type);struct map *map__new2 (u64 start,struct dso *dso,enum map_type type);void map__delete (struct map *map);
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 10/15 Revert "perf: Disable PERF_RECORD_MMAP2 support"
In reply to Jiri OlsaFrom: Don Zickus <[email protected]>
This reverts commit 3090ffb5a2515990182f3f55b0688a7817325488.
Re-enable the mmap2 interface as we will have a user soon.
Since things have changed since perf disabled mmap2, small tweaks to the revert had to be done:
o commit 9d4ecc88 forced (n!=8) to become (n<7)
o a new libunwind test needed updating to use mmap2 interface
--- kernel/events/core.c | 4 ---- tools/perf/tests/dwarf-unwind.c | 2 +- tools/perf/util/event.c | 34 ++++++++++++++++++++-------------- tools/perf/util/evsel.c | 1 + 4 files changed, 22 insertions(+), 19 deletions(-)
kernel/events/core.c +0 -4
@@ -6929,10 +6929,6 @@ static int perf_copy_attr(struct perf_event_attr __user *uattr,if (ret)return -EFAULT ; -/* disabled for now */ -if (attr->mmap2) -return -EINVAL ; -if (attr->__reserved_1)return -EINVAL ;
tools/perf/tests/dwarf-unwind.c +1 -1
@@ -15,7 +15,7 @@ static int mmap_handler(struct perf_tool *tool __maybe_unused,struct perf_sample *sample__maybe_unused , struct machine *machine) { - return machine__process_mmap_event(machine, event, NULL); + return machine__process_mmap2_event(machine, event, NULL); }static int init_live_machine (struct machine *machine)
tools/perf/util/event.c +20 -14
@@ -179,13 +179,14 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,return -1 ; } - event->header.type =PERF_RECORD_MMAP ; + event->header.type =PERF_RECORD_MMAP2 ;while (1 ) {char bf[BUFSIZ ];char prot[5 ];char execname[PATH_MAX ];char anonstr[] ="//anon" ; +unsigned int ino;size_t size;ssize_t n;
@@ -196,15 +197,20 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,strcpy (execname,"" );/* 00400000-0040c000 r-xp 00000000 fd:01 41038 /bin/cat */ - n =sscanf (bf,"%" PRIx64"-%" PRIx64" %s %" PRIx64" %*x:%*x %*u %s\n" , - &event->mmap.start, &event->mmap.len, prot, - &event->mmap.pgoff, - execname); + n =sscanf (bf,"%" PRIx64"-%" PRIx64" %s %" PRIx64" %x:%x %u %s\n" , + &event->mmap2.start, &event->mmap2.len, prot, + &event->mmap2.pgoff, &event->mmap2.maj, + &event->mmap2.min, + &ino, execname); +/* * Anon maps don't have the execname. */ -if (n <4 ) +if (n <7 )continue ; + + event->mmap2.ino = (u64 )ino; +/* * Just like the kernel, see __perf_event_mmap in kernel/perf_event.c */
@@ -239,15 +245,15 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,strcpy (execname, anonstr); size =strlen (execname) +1 ; -memcpy (event->mmap.filename, execname, size); +memcpy (event->mmap2.filename, execname, size); size =PERF_ALIGN (size,sizeof (u64)); - event->mmap.len -= event->mmap.start; - event->mmap.header.size = (sizeof (event->mmap) - - (sizeof (event->mmap.filename) - size)); -memset (event->mmap.filename + size,0 , machine->id_hdr_size); - event->mmap.header.size += machine->id_hdr_size; - event->mmap.pid = tgid; - event->mmap.tid = pid; + event->mmap2.len -= event->mmap.start; + event->mmap2.header.size = (sizeof (event->mmap2) - + (sizeof (event->mmap2.filename) - size)); +memset (event->mmap2.filename + size,0 , machine->id_hdr_size); + event->mmap2.header.size += machine->id_hdr_size; + event->mmap2.pid = tgid; + event->mmap2.tid = pid;if (process (tool, event, &synth_sample, machine) !=0 ) { rc =-1 ;
tools/perf/util/evsel.c +1 -0
@@ -659,6 +659,7 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)perf_evsel__set_sample_bit (evsel,WEIGHT ); attr->mmap = track; + attr->mmap2 = track && !perf_missing_features.mmap2; attr->comm = track;if (opts->sample_transaction)
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 11/15 perf report: Add mem-mode documentation to report command
In reply to Jiri OlsaFrom: Don Zickus <[email protected]>
Add mem-mode sorting types and mem-mode itself to perf-report documentation.
--- tools/perf/Documentation/perf-report.txt | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
tools/perf/Documentation/perf-report.txt +22 -0
@@ -117,6 +117,21 @@ OPTIONS By default, every sort keys not specified in -F will be appended automatically. + If --mem-mode option is used, following sort keys are also available + (incompatible with --branch-stack): + symbol_daddr, dso_daddr, locked, tlb, mem, snoop. + + - symbol_daddr: name of data symbol being executed on at the time of sample + - dso_daddr: name of library or module containing the data being executed + on at the time of sample + - locked: whether the bus was locked at the time of sample + - tlb: type of tlb access for the data at the time of sample + - mem: type of memory access for the data at the time of sample + - snoop: type of snoop (if any) for the data at the time of sample + + And default sort keys are changed to local_weight, mem, sym, dso, + symbol_daddr, dso_daddr, snoop, tlb, locked, see '--mem-mode'. + -p:: --parent=<regex>:: A regex filter to identify parent. The parent is a caller of this
@@ -260,6 +275,13 @@ OPTIONS Demangle symbol names to human readable form. It's enabled by default, disable with --no-demangle. +--mem-mode:: + Use the data addresses of samples in addition to instruction addresses + to build the histograms. To generate meaningful output, the perf.data + file must have been obtained using perf record -d -W and using a + special event -e cpu/mem-loads/ or -e cpu/mem-stores/. See + 'perf mem' for simpler access. + --percent-limit:: Do not show entries which have an overhead under that percent. (Default: 0).
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 12/15 perf tools: Add cpumode to struct hist_entry
In reply to Jiri OlsaFrom: Don Zickus <[email protected]>
The next patch needs to sort on cpumode, so add it to hist_entry to be tracked.
--- tools/perf/util/hist.c | 7 ++++--- tools/perf/util/sort.h | 1 + 2 files changed, 5 insertions(+), 3 deletions(-)
tools/perf/util/hist.c +4 -3
@@ -439,9 +439,10 @@ struct hist_entry *__hists__add_entry(struct hists *hists, .map = al->map, .sym = al->sym, }, - .cpu = al->cpu, - .ip = al->addr, - .level = al->level, + .cpu = al->cpu, + .cpumode = al->cpumode, + .ip = al->addr, + .level = al->level, .stat = { .nr_events = 1, .period = period,
tools/perf/util/sort.h +1 -0
@@ -89,6 +89,7 @@ struct hist_entry {u64 ip;u64 transaction;s32 cpu; +u8 cpumode;struct hist_entry_diff diff;
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 13/15 perf tools: Add support to dynamically get cacheline size
In reply to Jiri OlsaFrom: Don Zickus <[email protected]>
Different arches may have different cacheline sizes. Look it up and set a global variable for reference.
--- tools/perf/perf.c | 1 + tools/perf/util/util.c | 1 + tools/perf/util/util.h | 1 + 3 files changed, 3 insertions(+)
tools/perf/perf.c +1 -0
@@ -458,6 +458,7 @@ int main(int argc, const char **argv)/* The page_size is placed in util object. */ page_size =sysconf (_SC_PAGE_SIZE); + cacheline_size =sysconf (_SC_LEVEL1_DCACHE_LINESIZE); cmd =perf_extract_argv0_path (argv[0 ]);if (!cmd)
tools/perf/util/util.c +1 -0
@@ -17,6 +17,7 @@* XXX We need to find a better place for these things... */ unsigned int page_size; +int cacheline_size;bool test_attr__enabled;
tools/perf/util/util.h +1 -0
@@ -304,6 +304,7 @@ char *rtrim(char *s);void dump_stack (void );extern unsigned int page_size; +extern int cacheline_size;void get_term_dimensions (struct winsize *ws);
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 14/15 perf tools: Add dcacheline sort
In reply to Jiri OlsaFrom: Don Zickus <[email protected]>
In perf's 'mem-mode', one can get access to a whole bunch of details specific to a particular sample instruction. A bunch of those details relate to the data address.
One interesting thing you can do with data addresses is to convert them into a unique cacheline they belong too. Organizing these data cachelines into similar groups and sorting them can reveal cache contention.
This patch creates an alogorithm based on various sample details that can help group entries together into data cachelines and allows 'perf report' to sort on it.
The algorithm relies on having proper mmap2 support in the kernel to help determine if the memory map the data address belongs to is private to a pid or globally shared.
The alogortithm is as follows:
o group cpumodes together
o group entries with discovered maps together
o sort on major, minor, inode and inode generation numbers
o if userspace anon, then sort on pid
o sort on cachelines based on data addresses
The 'dcacheline' sort option in 'perf report' only works in 'mem-mode'.
Sample output:
#
# Samples: 206 of event 'cpu/mem-loads/pp'
# Total weight : 2534
# Sort order : dcacheline,pid
#
# Overhead Samples Data Cacheline Command: Pid
# ........ ............ ...................................................................... ..................
#
13.22% 1 [k] 0xffff88042f08ebc0 swapper: 0
9.27% 1 [k] 0xffff88082e8cea80 swapper: 0
3.59% 2 [k] 0xffffffff819ba180 swapper: 0
0.32% 1 [k] arch_trigger_all_cpu_backtrace_handler_na.23901+0xffffffffffffffe0 swapper: 0
0.32% 1 [k] timekeeper_seq+0xfffffffffffffff8 swapper: 0Note: Added a '+1' to symlen size in hists__calc_col_len to prevent the next column from prematurely tabbing over and mis-aligning. Not sure what the problem is.
--- tools/perf/Documentation/perf-report.txt | 3 +- tools/perf/util/hist.c | 2 + tools/perf/util/hist.h | 1 + tools/perf/util/sort.c | 107 +++++++++++++++++++++++++++++++ tools/perf/util/sort.h | 1 + 5 files changed, 113 insertions(+), 1 deletion(-)
tools/perf/Documentation/perf-report.txt +2 -1
@@ -119,7 +119,7 @@ OPTIONS If --mem-mode option is used, following sort keys are also available (incompatible with --branch-stack): - symbol_daddr, dso_daddr, locked, tlb, mem, snoop. + symbol_daddr, dso_daddr, locked, tlb, mem, snoop, dcacheline. - symbol_daddr: name of data symbol being executed on at the time of sample - dso_daddr: name of library or module containing the data being executed
@@ -128,6 +128,7 @@ OPTIONS - tlb: type of tlb access for the data at the time of sample - mem: type of memory access for the data at the time of sample - snoop: type of snoop (if any) for the data at the time of sample + - dcacheline: the cacheline the data address is on at the time of sample And default sort keys are changed to local_weight, mem, sym, dso, symbol_daddr, dso_daddr, snoop, tlb, locked, see '--mem-mode'.
tools/perf/util/hist.c +2 -0
@@ -128,6 +128,8 @@ void hists__calc_col_len(struct hists *hists, struct hist_entry *h) + unresolved_col_width +2 ;hists__new_col_len (hists,HISTC_MEM_DADDR_SYMBOL , symlen); +hists__new_col_len (hists,HISTC_MEM_DCACHELINE , + symlen +1 ); }else { symlen = unresolved_col_width +4 +2 ; hists__new_col_len(hists, HISTC_MEM_DADDR_SYMBOL,
tools/perf/util/hist.h +1 -0
@@ -72,6 +72,7 @@ enum hist_column {HISTC_MEM_TLB ,HISTC_MEM_LVL ,HISTC_MEM_SNOOP , +HISTC_MEM_DCACHELINE ,HISTC_TRANSACTION , HISTC_NR_COLS, /* Last entry */ };
tools/perf/util/sort.c +107 -0
128 lines not on this page · raw message
tools/perf/util/sort.h +1 -0
@@ -186,6 +186,7 @@ enum sort_type {SORT_MEM_TLB ,SORT_MEM_LVL ,SORT_MEM_SNOOP , + SORT_MEM_DCACHELINE, };/*
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 15/15 perf tools: Support spark lines in perf stat
In reply to Jiri OlsaFrom: Andi Kleen <[email protected]>
perf stat -rX prints the stddev for multiple measurements. Just looking at the stddev for judging the quality of the data is a bit dangerous The simplest sanity check is to just look at a simple plot. This patchs add a sparkline to the end of the measurements to make it simple to judge the data.
The sparkline only uses UTF-8, so should be readable in all modern tools and terminals.
The sparkline is between the minimum and maximum of the data, so it's mainly a indicator of variance. To keep the code simple and make the output not too wide only the first 8 values are printed. If more values are there it adds '..'
The code is inspired by Zach Holman's spark shell script.
Example output (view in non-proportial font):
Performance counter stats for 'true' (10 runs):
0.175672 task-clock (msec) # 0.555 CPUs utilized ( +- 1.77% ) █▄▁▁▁▁▁▁..
0 context-switches # 0.000 K/sec
0 cpu-migrations # 0.000 K/sec
114 page-faults # 0.647 M/sec ( +- 0.14% ) ▁█▁▁████..
520,798 cycles # 2.965 GHz ( +- 1.75% ) █▄▁▁▁▁▁▁..
433,525 instructions # 0.83 insns per cycle ( +- 0.28% ) ▅▇▅▄▇█▁▆..
83,012 branches # 472.537 M/sec ( +- 0.31% ) ▅▇▆▄▇█▁▆..
3,157 branch-misses # 3.80% of all branches ( +- 2.55% ) ▇█▃▅▁▃▁▂..0.000316660 seconds time elapsed ( +- 1.78% ) █▅▁▁▁▁▁▁..
As you can see even in the most simple run there are quite interesting patterns. The time sparkline suggests it would be also useful to have an option to throw the first measurement away.
Known issues:
- Makes the perf stat output wider. Could be adjust by shrinking some white space. Not done so far.
- No output for -A/--per-socket/--per-core with -rX. This code is missing the basic noise detection code. Once it's added there sparklines could be shown too.
--- tools/perf/Documentation/perf-stat.txt | 4 ++++ tools/perf/Makefile.perf | 1 + tools/perf/builtin-stat.c | 12 ++++++++++++ tools/perf/util/spark.c | 31 +++++++++++++++++++++++++++++++ tools/perf/util/spark.h | 4 ++++ tools/perf/util/stat.c | 34 ++++++++++++++++++++++++++++++++++ tools/perf/util/stat.h | 10 ++++++++++ 7 files changed, 96 insertions(+) create mode 100644 tools/perf/util/spark.c create mode 100644 tools/perf/util/spark.h
tools/perf/Documentation/perf-stat.txt +4 -0
@@ -53,6 +53,10 @@ OPTIONS -r:: --repeat=<n>:: repeat command and print average + stddev (max: 100). 0 means forever. + In addition it prints a spark line (when not in CSV mode), which visualizes the + variance between minimum and maximum of the measurements. This allows a simple sanity + check of the measurements. Only 8 values are printed, when more are available + it adds .. -B:: --big-num::
tools/perf/Makefile.perf +1 -0
@@ -359,6 +359,7 @@ LIB_OBJS += $(OUTPUT)util/trace-event-scripting.o LIB_OBJS += $(OUTPUT)util/trace-event.o LIB_OBJS += $(OUTPUT)util/svghelper.o LIB_OBJS += $(OUTPUT)util/sort.o +LIB_OBJS += $(OUTPUT)util/spark.o LIB_OBJS += $(OUTPUT)util/hist.o LIB_OBJS += $(OUTPUT)util/probe-event.o LIB_OBJS += $(OUTPUT)util/util.o
tools/perf/builtin-stat.c +12 -0
@@ -1176,6 +1176,9 @@ static void print_aggr(char *prefix)if (run != ena)fprintf (output," (%.2f%%)" ,100.0 * run / ena); + +fputc (' ' , output); +print_stat_spark (output, counter->priv); }fputc ('\n' , output); }
@@ -1229,6 +1232,9 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)return ; } +fputc (' ' , output); +print_stat_spark (output, counter->priv); +if (scaled) {double avg_enabled, avg_running;
@@ -1295,6 +1301,9 @@ static void print_counter(struct perf_evsel *counter, char *prefix)if (run != ena)fprintf (output," (%.2f%%)" ,100.0 * run / ena); + +fputc (' ' , output); +print_stat_spark (output, counter->priv); }fputc ('\n' , output); }
@@ -1355,6 +1364,9 @@ static void print_stat(int argc, const char **argv)fprintf (output," " );print_noise_pct (stddev_stats (&walltime_nsecs_stats),avg_stats (&walltime_nsecs_stats)); + +fputc (' ' , output); +print_stat_spark (output, &walltime_nsecs_stats); }fprintf (output,"\n\n" ); }
tools/perf/util/spark.c new file +31 -0
@@ -0,0 +1,31 @@ +#include <stdio.h> +#include <limits.h> +#include "spark.h" + +#define NUM_SPARKS 8 +#define SPARK_SHIFT 8 + +/* Print spark lines on outf for numval values in val. */ +void print_spark (FILE *outf,unsigned long long *val,int numval) +{ +static const char *ticks[NUM_SPARKS ] = { +"▁" ,"▂" ,"▃" ,"▄" ,"▅" ,"▆" ,"▇" ,"█" + }; +int i; +unsigned long long min =ULLONG_MAX , max =0 , f; + +for (i =0 ; i < numval; i++) { +if (val[i] < min) + min = val[i]; +if (val[i] > max) + max = val[i]; + } + f = ((max - min) <<SPARK_SHIFT ) / (NUM_SPARKS -1 ); +if (f <1 ) + f =1 ; +for (i =0 ; i < numval; i++) { +int index = ((val[i] - min) <<SPARK_SHIFT ) / f; +if (index >=0 && index <NUM_SPARKS ) +fputs (ticks[index], outf); + } +}
tools/perf/util/spark.h new file +4 -0
@@ -0,0 +1,4 @@ +#ifndef SPARK_H +#define SPARK_H 1 +void print_spark (FILE *outf,unsigned long long *val,int numval); +#endif
tools/perf/util/stat.c +34 -0
@@ -1,10 +1,16 @@#include <math.h> +#include <stdio.h> #include "stat.h" +#include "spark.h" void update_stats (struct stats *stats,u64 val) {double delta; +int n = stats->n; + +if (n <NUM_SPARK_VALS ) + stats->svals[n] = val; stats->n++; delta = val - stats->mean;
@@ -61,3 +67,31 @@ double rel_stddev_stats(double stddev, double avg)return pct; } + +static int all_zero (unsigned long long *vals,int len) +{ +int i; + +for (i =0 ; i < len; i++) +if (vals[i] !=0 ) +return 0 ; +return 1 ; +} + +void print_stat_spark (FILE *f,struct stats *stat) +{ +int len; + +if (stat->n <=1 ) +return ; + + len = stat->n; +if (len >NUM_SPARK_VALS ) + len =NUM_SPARK_VALS ; +if (all_zero (stat->svals, len)) +return ; + +print_spark (f, stat->svals, len); +if (stat->n >NUM_SPARK_VALS ) +fputs (".." , f); +}
tools/perf/util/stat.h +10 -0
@@ -1,12 +1,16 @@#ifndef __PERF_STATS_H#define __PERF_STATS_H +#include <stdio.h> #include <linux/types.h> +#define NUM_SPARK_VALS 8/* support spark line on first N items */ +struct stats {double n, mean, M2;u64 max, min; +unsigned long long svals[NUM_SPARK_VALS ]; };void update_stats (struct stats *stats,u64 val);
@@ -14,12 +18,18 @@ double avg_stats(struct stats *stats);double stddev_stats (struct stats *stats);double rel_stddev_stats (double stddev,double avg); +void print_stat_spark (FILE *f,struct stats *stat); +static inline void init_stats (struct stats *stats) { +int i; + stats->n =0.0 ; stats->mean =0.0 ; stats->M2 =0.0 ; stats->min = (u64) -1 ; stats->max =0 ; +for (i =0 ; i <NUM_SPARK_VALS ; i++) + stats->svals[i] =0 ; }#endif
Signature · 1 line
1.8.3.1
Re: [PATCH 15/15] perf tools: Support spark lines in perf stat
Quoted 3 lines · Jiri Olsa
From: Andi Kleen <[email protected]>
perf stat -rX prints the stddev for multiple measurements.
Call it "--repeat X", that's how most people know it.
Quoted 2 lines
Just looking at the stddev for judging the quality of the data is a bit dangerous The simplest sanity check is to just look
"Dangerous?". Unexplained FUD. Also a missing perid.
Quoted 2 lines
at a simple plot. This patchs add a sparkline to the end of the measurements to make it simple to judge the data.
Should explain what a 'sparkline' is when it's first mentioned, not 2 sentences later.
Quoted 9 lines
The sparkline only uses UTF-8, so should be readable in all modern tools and terminals.
The sparkline is between the minimum and maximum of the data, so it's mainly a indicator of variance. To keep the code simple and make the output not too wide only the first 8 values are printed. If more values are there it adds '..'
The code is inspired by Zach Holman's spark shell script.
Zach Holman should be on Cc:.
Quoted 13 lines
Example output (view in non-proportial font):
Performance counter stats for 'true' (10 runs):
0.175672 task-clock (msec) # 0.555 CPUs utilized ( +- 1.77% ) █▄▁▁▁▁▁▁.. 0 context-switches # 0.000 K/sec 0 cpu-migrations # 0.000 K/sec 114 page-faults # 0.647 M/sec ( +- 0.14% ) ▁█▁▁████.. 520,798 cycles # 2.965 GHz ( +- 1.75% ) █▄▁▁▁▁▁▁.. 433,525 instructions # 0.83 insns per cycle ( +- 0.28% ) ▅▇▅▄▇█▁▆.. 83,012 branches # 472.537 M/sec ( +- 0.31% ) ▅▇▆▄▇█▁▆.. 3,157 branch-misses # 3.80% of all branches ( +- 2.55% ) ▇█▃▅▁▃▁▂..
Quoted 1 line
0.000316660 seconds time elapsed ( +- 1.78% ) █▅▁▁▁▁▁▁..
The noise data is not aligned vertically in the changelog - is it in the real code? It should be.
Quoted 3 lines
As you can see even in the most simple run there are quite interesting patterns. The time sparkline suggests it would be also useful to have an option to throw the first measurement away.
First two should be thrown away, probably, to reduce the resulting stddev.
Quoted 6 lines
Known issues:
- Makes the perf stat output wider. Could be adjust by shrinking some white space. Not done so far.
- No output for -A/--per-socket/--per-core with -rX. This code is missing the basic noise detection code. Once it's added there sparklines could be shown too.
The feauture looks useful, but at minimum the second shortcoming should be fixed first in a separate patch, before applying new features and further complicating the code.
Quoted 77 lines
Signed-off-by Andi Kleen <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/perf/Documentation/perf-stat.txt | 4 ++++ tools/perf/Makefile.perf | 1 + tools/perf/builtin-stat.c | 12 ++++++++++++ tools/perf/util/spark.c | 31 +++++++++++++++++++++++++++++++ tools/perf/util/spark.h | 4 ++++ tools/perf/util/stat.c | 34 ++++++++++++++++++++++++++++++++++ tools/perf/util/stat.h | 10 ++++++++++ 7 files changed, 96 insertions(+) create mode 100644 tools/perf/util/spark.c create mode 100644 tools/perf/util/spark.htools/perf/Documentation/perf-stat.txt +4
-0@@ -53,6 +53,10 @@ OPTIONS -r:: --repeat=<n>:: repeat command and print average + stddev (max: 100). 0 means forever. + In addition it prints a spark line (when not in CSV mode), which visualizes the + variance between minimum and maximum of the measurements. This allows a simple sanity + check of the measurements. Only 8 values are printed, when more are available + it adds .. -B:: --big-num::tools/perf/Makefile.perf +1
-0@@ -359,6 +359,7 @@ LIB_OBJS += $(OUTPUT)util/trace-event-scripting.o LIB_OBJS += $(OUTPUT)util/trace-event.o LIB_OBJS += $(OUTPUT)util/svghelper.o LIB_OBJS += $(OUTPUT)util/sort.o +LIB_OBJS += $(OUTPUT)util/spark.o LIB_OBJS += $(OUTPUT)util/hist.o LIB_OBJS += $(OUTPUT)util/probe-event.o LIB_OBJS += $(OUTPUT)util/util.otools/perf/builtin-stat.c +9
-0@@ -1176,6 +1176,9 @@ static void print_aggr(char *prefix)if (run != ena)fprintf (output," (%.2f%%)" ,100.0 * run / ena); + +fputc (' ' , output); +print_stat_spark (output, counter->priv); }fputc ('\n' , output); }@@ -1229,6 +1232,9 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)return ; } +fputc (' ' , output); +print_stat_spark (output, counter->priv); +if (scaled) {double avg_enabled, avg_running;@@ -1295,6 +1301,9 @@ static void print_counter(struct perf_evsel *counter, char *prefix)if (run != ena)fprintf (output," (%.2f%%)" ,100.0 * run / ena); + +fputc (' ' , output); +print_stat_spark (output, counter->priv); }fputc ('\n' , output); }
tools/perf/builtin-stat.c static void print_stat(int argc, const char **argv) · line 1369
Quoted 6 lines
tools/perf/builtin-stat.c +3
-0@@ -1355,6 +1364,9 @@ static void print_stat(int argc, const char **argv)fprintf (output," " );print_noise_pct (stddev_stats (&walltime_nsecs_stats),avg_stats (&walltime_nsecs_stats)); + +fputc (' ' , output); +print_stat_spark (output, &walltime_nsecs_stats);
Why is that 2-line pattern repeated 4 times?
Quoted 9 lines
tools/perf/builtin-stat.c +0
-0}fprintf (output,"\n\n" ); }tools/perf/util/spark.c new file +0
-0
tools/perf/util/spark.c line 9
Quoted 9 lines
tools/perf/util/spark.c new file +9
-0@@ -0,0 +1,31 @@ +#include <stdio.h> +#include <limits.h> +#include "spark.h" + +#define NUM_SPARKS 8 +#define SPARK_SHIFT 8 + +/* Print spark lines on outf for numval values in val. */ +void print_spark (FILE *outf,unsigned long long *val,int numval)
The output file is called 'output' in the calling context, there's no reason to randomly depart from that convention and introduce a random new one: not a single existing line in tools/perf/ calls an output file 'outf'.
Also, I don't see 'numval' used anywhere in perf. The canonical naming would be different.
tools/perf/util/spark.c line 15
Quoted 6 lines
tools/perf/util/spark.c new file +6
-0+{ +static const char *ticks[NUM_SPARKS ] = { +"▁" ,"▂" ,"▃" ,"▄" ,"▅" ,"▆" ,"▇" ,"█" + }; +int i; +unsigned long long min =ULLONG_MAX , max =0 , f;
min/max is a somewhat sloppy name as well (easy to confuse with min()/max()), should be somehing like min_val/max_val, which is in line .
Quoted 4 lines
tools/perf/util/spark.c new file +4
-0+ +for (i =0 ; i < numval; i++) { +if (val[i] < min) + min = val[i];
tools/perf/util/spark.c line 29
Quoted 10 lines
tools/perf/util/spark.c new file +10
-0+if (val[i] > max) + max = val[i]; + } + f = ((max - min) <<SPARK_SHIFT ) / (NUM_SPARKS -1 ); +if (f <1 ) + f =1 ; +for (i =0 ; i < numval; i++) { +int index = ((val[i] - min) <<SPARK_SHIFT ) / f; +if (index >=0 && index <NUM_SPARKS ) +fputs (ticks[index], outf);
That's not primarily an 'index', it's an indexed 'variance' value. So 'variance_idx' or so.
Quoted 33 lines
tools/perf/util/spark.c new file +2
-0+ } +}tools/perf/util/spark.h new file +4
-0@@ -0,0 +1,4 @@ +#ifndef SPARK_H +#define SPARK_H 1 +void print_spark (FILE *outf,unsigned long long *val,int numval); +#endif tools/perf/util/stat.c +6
-0@@ -1,10 +1,16 @@#include <math.h> +#include <stdio.h> #include "stat.h" +#include "spark.h" void update_stats (struct stats *stats,u64 val) {double delta; +int n = stats->n; + +if (n <NUM_SPARK_VALS ) + stats->svals[n] = val; stats->n++; delta = val - stats->mean;
Quoted 1 line
stats->mean += delta / stats->n;
It is absolutely sloppy, potentially misleading and unrobust to put an integer rounded value of 'double n' into a local variable named 'int n' ...
Quoted 41 lines
tools/perf/util/stat.c +28
-0@@ -61,3 +67,31 @@ double rel_stddev_stats(double stddev, double avg)return pct; } + +static int all_zero (unsigned long long *vals,int len) +{ +int i; + +for (i =0 ; i < len; i++) +if (vals[i] !=0 ) +return 0 ; +return 1 ; +} + +void print_stat_spark (FILE *f,struct stats *stat) +{ +int len; + +if (stat->n <=1 ) +return ; + + len = stat->n; +if (len >NUM_SPARK_VALS ) + len =NUM_SPARK_VALS ; +if (all_zero (stat->svals, len)) +return ; + +print_spark (f, stat->svals, len); +if (stat->n >NUM_SPARK_VALS ) +fputs (".." , f); +}tools/perf/util/stat.h +1
-0@@ -1,12 +1,16 @@#ifndef __PERF_STATS_H#define __PERF_STATS_H +#include <stdio.h>
tools/perf/util/stat.h line 14
Quoted 10 lines
tools/perf/util/stat.h +3
-0#include <linux/types.h> +#define NUM_SPARK_VALS 8/* support spark line on first N items */ +struct stats {double n, mean, M2;u64 max, min; +unsigned long long svals[NUM_SPARK_VALS ]; };
The structure's increasing size justifies proper vertical alignment.
Quoted 9 lines
tools/perf/util/stat.h +2
-0void update_stats (struct stats *stats,u64 val);@@ -14,12 +18,18 @@ double avg_stats(struct stats *stats);double stddev_stats (struct stats *stats);double rel_stddev_stats (double stddev,double avg); +void print_stat_spark (FILE *f,struct stats *stat); +static inline void init_stats (struct stats *stats)
tools/perf/util/stat.h double avg_stats(struct stats *stats); · line 33
Quoted 10 lines
tools/perf/util/stat.h +4
-0{ +int i; + stats->n =0.0 ; stats->mean =0.0 ; stats->M2 =0.0 ; stats->min = (u64) -1 ; stats->max =0 ; +for (i =0 ; i <NUM_SPARK_VALS ; i++) + stats->svals[i] =0 ;
memset(), or introduce a new helper, memclear_struct():
#define memclear_struct(struct_ptr) memset(struct_ptr, 0, sizeof(*(struct_ptr)))
Also, init_stats() should now probably move into util/stat.c. (in a separate patch preparing these changes.)
Thanks,
Ingo
Peter Zijlstra
maintainerRe: [PATCH 15/15] perf tools: Support spark lines in perf stat
Quoted 13 lines · Ingo Molnar
Quoted 13 lines
Example output (view in non-proportial font):
Performance counter stats for 'true' (10 runs):
0.175672 task-clock (msec) # 0.555 CPUs utilized ( +- 1.77% ) █▄▁▁▁▁▁▁.. 0 context-switches # 0.000 K/sec 0 cpu-migrations # 0.000 K/sec 114 page-faults # 0.647 M/sec ( +- 0.14% ) ▁█▁▁████.. 520,798 cycles # 2.965 GHz ( +- 1.75% ) █▄▁▁▁▁▁▁.. 433,525 instructions # 0.83 insns per cycle ( +- 0.28% ) ▅▇▅▄▇█▁▆.. 83,012 branches # 472.537 M/sec ( +- 0.31% ) ▅▇▆▄▇█▁▆.. 3,157 branch-misses # 3.80% of all branches ( +- 2.55% ) ▇█▃▅▁▃▁▂..
Quoted 4 lines
Quoted 1 line
0.000316660 seconds time elapsed ( +- 1.78% ) █▅▁▁▁▁▁▁..
The noise data is not aligned vertically in the changelog - is it in the real code? It should be.
I still dislike those spark lines, I still think they should plot the distribution and not just the first 8 or so values.
Quoted 101 lines · Jiri Olsa
hi Ingo,
please consider pullingthanks,
jirkaThe following changes since commit
82b897782d10fcc4930c9d4a15b175348fdd2871:perf: Differentiate
exec()and non-exec() comm events (2014-06-06 07:56:22 +0200)are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.gittags/perf-core-for-mingofor you to fetch changes up to
a2609f3b0c582d6aaa8f69a61a0eea6c7a98d291:perf tools: Support spark lines in perf stat (2014-06-09 13:34:50 +0200)
----------------------------------------------------------------
perf/core improvements and fixes:. Bitmask handling and plugin updates (Steven Rostedt)
. Fix pipe check regression in attr event callback (Jiri Olsa)
. Prettify the tags/TAGS/cscope targets output (Jiri Olsa)
. Print array argument as string (Namhyung Kim)
. Pass protection and flags bits through mmap2 interface (Peter Zijlstra)
. Update perf tool mmap2 interface with protection and flag bits (Don Zickus)
. Re-enable mmap interface (Don Zickus)
. Add mem-mode documentation to report command (Don Zickus)
. Add sort on dcacheline (Don Zickus)
. Support spark lines in perf stat (Andi Kleen)
Signed-off-by Jiri Olsa <[email protected]>----------------------------------------------------------------
Andi Kleen (1):
perf tools: Support spark lines in perf statDon Zickus (6):
perf tools: Update mmap2 interface with protection and flag bits Revert "perf: DisablePERF_RECORD_MMAP2support"
perf report: Add mem-mode documentation to report command perf tools: Add cpumode to structhist_entry
perf tools: Add support to dynamically get cacheline size perf tools: Add dcacheline sortJiri Olsa (2):
perf tools: Fix pipe check regression in attr event callback perf tools: Prettify the tags/TAGS/cscope targets outputNamhyung Kim (1):
perf script/python: Print array argument as stringPeter Zijlstra (1):
perf: Pass protection and flags bits through mmap2 interfaceSteven Rostedt (1):
tools lib traceevent: Add options to pluginsSteven Rostedt (Red Hat) (3):
tools lib traceevent: Add flag to not load event plugins tools lib traceevent: Add options to function plugin tools lib traceevent: Added support for__get_bitmask()macroinclude/uapi/linux/perf_event.h | 1 + kernel/events/core.c | 37 +++- tools/lib/traceevent/event-parse.c | 113 ++++++++++++ tools/lib/traceevent/event-parse.h | 25 ++- tools/lib/traceevent/event-plugin.c | 203 ++++++++++++++++++++- tools/lib/traceevent/plugin_function.c | 43 ++++- tools/perf/Documentation/perf-report.txt | 23 +++ tools/perf/Documentation/perf-stat.txt | 4 + tools/perf/Makefile.perf | 7 +- tools/perf/builtin-inject.c | 2 +- tools/perf/builtin-stat.c | 12 ++ tools/perf/perf.c | 1 + tools/perf/tests/dwarf-unwind.c | 2 +- tools/perf/util/event.c | 57 ++++-- tools/perf/util/event.h | 2 + tools/perf/util/evsel.c | 1 + tools/perf/util/hist.c | 9 +- tools/perf/util/hist.h | 1 + tools/perf/util/machine.c | 4 +- tools/perf/util/map.c | 4 +- tools/perf/util/map.h | 4 +- .../perf/util/scripting-engines/trace-event-perl.c | 1 + .../util/scripting-engines/trace-event-python.c | 2 + tools/perf/util/sort.c | 107 +++++++++++ tools/perf/util/sort.h | 2 + tools/perf/util/spark.c | 31 ++++
Quoted 8 lines
tools/perf/util/spark.h | 4 + tools/perf/util/stat.c | 34 ++++ tools/perf/util/stat.h | 10 + tools/perf/util/util.c | 1 + tools/perf/util/util.h | 1 + 31 files changed, 707 insertions(+), 41 deletions(-) create mode 100644 tools/perf/util/spark.c create mode 100644 tools/perf/util/spark.h
Pulled, thanks a lot Jiri!
Ingo