perf: Add support for full Intel event lists v7
Should be ready for merge now. Please consider.
[v2: Review feedback addressed and some minor improvements] [v3: More review feedback addressed and handle test failures better. Ported to latest tip/core.]
[v4: Addressed Namhyung's feedback]
[v5: Rebase to latest tree. Minor description update.]
[v6: Rebase. Add acked by from Namhyung and address feedback. Some minor fixes. Should be good to go now I hope. The period patch was dropped, as that is already handled. I added an extra patch for a --quiet argument for perf list]
[v7: Just rebase to latest tip/core. Should be ready to merge.]
perf has high level events which are useful in many cases. However there are some tuning situations where low level events in the CPU are needed. Traditionally this required specifying the event in raw form (very awkward) or using non standard frontends like ocperf or patching in libpfm.
Intel CPUs can have very large event files (Haswell has ~336 core events, much more if you add uncore or all the offcore combinations), which is too large to describe through the kernel interface. It would require tying up significant amounts of unswappable memory for this.
oprofile always had separate event list files that were maintained by the CPU vendors. The oprofile events were shipped with the tool. The Intel events get updated regularly, for example to add references to the specification updates or add new events.
Unfortunately oprofile usually did not keep up with these updates, so the events in oprofile were often out of date. In addition it ties up quite a bit of disk space, mostly for CPUs you don't have.
This patch kit implements another mechanism that avoids these problems. Intel releases the event lists for CPUs in a standardized JSON format on a download server.
I implemented an automatic downloader to get the event file for the current CPU. The events are stored in ~/.cache/pmu-events. Then perf adds a parser that converts the JSON format into perf event aliases, which then can be used directly as any other perf event.
The parsing is done using a simple existing JSON library.
The events are still abstracted for perf, but the abstraction mechanism is through the downloaded file instead of through the kernel.
The JSON format and perf parser has some minor Intelisms, but they are simple and small and optional. It's easy to extend, so it would be possible to use it for other CPUs too, add different pmu attributes, and add new download sites to the downloader tool.
Currently only core events are supported, uncore may come at a later point. No kernel changes, all code in perf user tools only.
Some of the parser files are partially shared with separate event parser library and are thus 2-clause BSD licensed.
Patches also available fromgit://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc perf/json
Example output:
% perf download
Downloading models file
Downloading readme.txt
2014-03-05 10:39:33 URL:https://download.01.org/perfmon/readme.txt [10320/10320] -> "readme.txt" [1] 2014-03-05 10:39:34 URL:https://download.01.org/perfmon/mapfile.csv [1207/1207] -> "mapfile.csv" [1]
Downloading events file
% perf list
...
br_inst_exec.all_branches [Speculative and retired
branches]
br_inst_exec.all_conditional [Speculative and retired
macro-conditional branches]
br_inst_exec.all_direct_jmp [Speculative and retired
macro-unconditional branches excluding calls and indirects]
... 333 more new events ...
% perf stat -e br_inst_exec.all_direct_jmp true
Performance counter stats for 'true':
6,817 cpu/br_inst_exec.all_direct_jmp/
0.003503212 seconds time elapsedOne nice feature is that a pointer to the specification update is now included in the description, which will hopefully clear up many problems:
% perf list
...
mem_load_uops_l3_hit_retired.xsnp_hit [Retired load uops which
data sources were L3 and cross-core snoop hits in on-pkg core cache. Supports address when precise. Spec
update: HSM26, HSM30 (Precise event)]
...
-Andi
PATCH 1/9 perf, tools: Add jsmn `jasmine' JSON parser v3
From: Andi Kleen <[email protected]>
I need a JSON parser. This adds the simplest JSON parser I could find -- Serge Zaitsev's jsmn `jasmine' -- to the perf library. I merely converted it to (mostly) Linux style and added support for non 0 terminated input.
The parser is quite straight forward and does not copy any data, just returns tokens with offsets into the input buffer. So it's relatively efficient and simple to use.
The code is not fully checkpatch clean, but I didn't want to completely fork the upstream code.
Original source: http://zserge.bitbucket.org/jsmn.html
In addition I added a simple wrapper that mmaps a json file and provides some straight forward access functions.
Used in follow-on patches to parse event files.
v2: Address review feedback.
v3: Minor checkpatch fixes.
--- tools/perf/Makefile.perf | 4 + tools/perf/util/jsmn.c | 313 +++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/jsmn.h | 67 ++++++++++ tools/perf/util/json.c | 155 +++++++++++++++++++++++ tools/perf/util/json.h | 13 ++ 5 files changed, 552 insertions(+) create mode 100644 tools/perf/util/jsmn.c create mode 100644 tools/perf/util/jsmn.h create mode 100644 tools/perf/util/json.c create mode 100644 tools/perf/util/json.h
tools/perf/Makefile.perf +4 -0
@@ -300,6 +300,8 @@ LIB_H += ui/progress.h LIB_H += ui/util.h LIB_H += ui/ui.h LIB_H += util/data.h +LIB_H += util/jsmn.h +LIB_H += util/json.h LIB_OBJS += $(OUTPUT)util/abspath.o LIB_OBJS += $(OUTPUT)util/alias.o
@@ -373,6 +375,8 @@ LIB_OBJS += $(OUTPUT)util/stat.o LIB_OBJS += $(OUTPUT)util/record.o LIB_OBJS += $(OUTPUT)util/srcline.o LIB_OBJS += $(OUTPUT)util/data.o +LIB_OBJS += $(OUTPUT)util/jsmn.o +LIB_OBJS += $(OUTPUT)util/json.o LIB_OBJS += $(OUTPUT)ui/setup.o LIB_OBJS += $(OUTPUT)ui/helpline.o
tools/perf/util/jsmn.c new file +313 -0
313 lines not on this page · raw message
tools/perf/util/jsmn.h new file +67 -0
67 lines not on this page · raw message
tools/perf/util/json.c new file +155 -0
155 lines not on this page · raw message
tools/perf/util/json.h new file +13 -0
@@ -0,0 +1,13 @@ +#ifndef JSON_H +#define JSON_H 1 + +#include "jsmn.h" + +jsmntok_t *parse_json (const char *fn,char **map,size_t *size,int *len); +void free_json (char *map,size_t size,jsmntok_t *tokens); +int json_line (char *map,jsmntok_t *t); +const char *json_name (jsmntok_t *t); +int json_streq (char *map,jsmntok_t *t,const char *s); +int json_len (jsmntok_t *t); + +#endif
Signature · 1 line
1.9.3
PATCH 2/9 perf, tools: Add support for text descriptions of events and alias add
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Change pmu.c to allow descriptions of events and add interfaces to add aliases at runtime from another file. To be used by jevents in the next patch.
--- tools/perf/util/pmu.c | 127 ++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 98 insertions(+), 29 deletions(-)
tools/perf/util/pmu.c +98 -29
@@ -14,6 +14,7 @@struct perf_pmu_alias {char *name; +char *desc;struct list_head terms;struct list_head list;char unit[UNIT_MAX_LEN +1 ];
@@ -171,17 +172,12 @@ error:return -1 ; } -static int perf_pmu__new_alias (struct list_head *list,char *dir,char *name,FILE *file) +static int __perf_pmu__new_alias (struct list_head *list,char *name, +char *dir,char *desc,char *val) {struct perf_pmu_alias *alias; -char buf[256 ];int ret; - ret =fread (buf,1 ,sizeof (buf), file); -if (ret ==0 ) -return -EINVAL ; - buf[ret] =0 ; - alias =malloc (sizeof (*alias));if (!alias)return -ENOMEM ;
@@ -190,24 +186,45 @@ static int perf_pmu__new_alias(struct list_head *list, char *dir, char *name, FI alias->scale =1.0 ; alias->unit[0 ] ='\0' ; - ret =parse_events_terms (&alias->terms, buf); + ret =parse_events_terms (&alias->terms, val);if (ret) { +pr_err ("Cannot parse alias %s: %d\n" , val, ret);free (alias);return ret; } alias->name =strdup (name); -/* -* load unit name and scale if available -*/ -perf_pmu__parse_unit (alias, dir, name); -perf_pmu__parse_scale (alias, dir, name); +if (dir) { +/* +* load unit name and scale if available +*/ +perf_pmu__parse_unit (alias, dir, name); +perf_pmu__parse_scale (alias, dir, name); + } + + alias->desc = desc ?strdup (desc) :NULL ;list_add_tail (&alias->list, list);return 0 ; } +static int perf_pmu__new_alias (struct list_head *list, +char *dir, +char *name, +FILE *file) +{ +char buf[256 ]; +int ret; + + ret =fread (buf,1 ,sizeof (buf), file); +if (ret ==0 ) +return -EINVAL ; + buf[ret] =0 ; + +return __perf_pmu__new_alias (list, name, dir,NULL , buf); +} +/* * Process all the sysfs attributes located under the directory * specified in 'dir' parameter.
@@ -720,11 +737,51 @@ static char *format_alias_or(char *buf, int len, struct perf_pmu *pmu,return buf; } -static int cmp_string (const void *a,const void *b) +struct pair { +char *name; +char *desc; +}; + +static int cmp_pair (const void *a,const void *b) +{ +const struct pair *as = a; +const struct pair *bs = b; + +/* Put downloaded event list last */ +if (!!as->desc != !!bs->desc) +return !!as->desc - !!bs->desc; +return strcmp (as->name, bs->name); +} + +static void wordwrap (char *s,int start,int max,int corr) { -const char *const *as = a; -const char *const *bs = b; -return strcmp (*as, *bs); +int column = start; +int n; + +while (*s) { +int wlen =strcspn (s," \t" ); + +if (column + wlen >= max && column > start) { +printf ("\n%*s" , start,"" ); + column = start + corr; + } + n =printf ("%s%.*s" , column > start ?" " :"" , wlen, s); +if (n <=0 ) +break ; + s += wlen; + column += n; +while (isspace (*s)) + s++; + } +} + +static int get_columns (void ) +{ +/* +* Should ask the terminal with TIOCGWINSZ here, but we +* need the original fd before the pager. +*/ +return 79 ; }void print_pmu_events (const char *event_glob,bool name_only)
@@ -734,21 +791,24 @@ void print_pmu_events(const char *event_glob, bool name_only)char buf[1024 ];int printed =0 ;int len, j; -char **aliases; +struct pair *aliases; +int numdesc =0 ; +int columns =get_columns (); pmu =NULL ; len =0 ;while ((pmu =perf_pmu__scan (pmu)) !=NULL )list_for_each_entry (alias, &pmu->aliases, list) len++; - aliases =malloc (sizeof (char *) * len); + aliases =malloc (sizeof (struct pair ) * len);if (!aliases)return ; pmu =NULL ; j =0 ;while ((pmu =perf_pmu__scan (pmu)) !=NULL )list_for_each_entry (alias, &pmu->aliases, list) { -char *name =format_alias (buf,sizeof (buf), pmu, alias); +char *name = alias->desc ? alias->name : +format_alias (buf,sizeof (buf), pmu, alias);bool is_cpu = !strcmp (pmu->name,"cpu" ); if (event_glob != NULL &&
@@ -756,22 +816,31 @@ void print_pmu_events(const char *event_glob, bool name_only) (!is_cpu &&strglobmatch (alias->name, event_glob)))) continue; - aliases[j] = name; -if (is_cpu && !name_only) - aliases[j] =format_alias_or (buf,sizeof (buf), - pmu, alias); - aliases[j] =strdup (aliases[j]); + aliases[j].name = name; +if (is_cpu && !name_only && !alias->desc) + aliases[j].name =format_alias_or (buf, +sizeof (buf), + pmu, alias); + aliases[j].name =strdup (aliases[j].name); + aliases[j].desc = alias->desc; j++; } len = j; -qsort (aliases, len,sizeof (char *), cmp_string); +qsort (aliases, len,sizeof (struct pair ), cmp_pair);for (j =0 ; j < len; j++) {if (name_only) { -printf ("%s " , aliases[j]); +printf ("%s " , aliases[j].name);continue ; } -printf (" %-50s [Kernel PMU event]\n" , aliases[j]); -zfree (&aliases[j]); +if (aliases[j].desc) { +if (numdesc++ ==0 && printed) +printf ("\n" ); +printf (" %-50s [" , aliases[j].name); +wordwrap (aliases[j].desc,53 , columns,1 ); +printf ("]\n" ); + }else +printf (" %-50s [Kernel PMU event]\n" , aliases[j].name); +zfree (&aliases[j].name); printed++; }if (printed)
Signature · 1 line
1.9.3
Re: [PATCH 2/9] perf, tools: Add support for text descriptions of events and alias add
Quoted 9 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Change
pmu.cto allow descriptions of events and add interfaces to add aliases at runtime from another file. To be used by jevents in the next patch.Acked-by Namhyung Kim <[email protected]>Signed-off-by Andi Kleen <[email protected]>---
SNIP
tools/perf/util/pmu.c static char *format_alias_or(char *buf, int len, struct perf_pmu *pmu, · line 785
Quoted 10 lines
tools/perf/util/pmu.c +9
-0+} + +static int get_columns (void ) +{ +/* +* Should ask the terminal with TIOCGWINSZ here, but we +* need the original fd before the pager. +*/ +return 79 ; }
so this ^^^ gets removed by
perf, tools: Query terminal width and use in perf list
please move that patch ahead of this one
jirka
Re: [PATCH 2/9] perf, tools: Add support for text descriptions of events and alias add
Quoted 4 lines
so this ^^^ gets removed by
perf, tools: Query terminal width and use in perf listplease move that patch ahead of this one
This cannot be done. There is no code to consume the column width before the other patch.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Re: [PATCH 2/9] perf, tools: Add support for text descriptions of events and alias add
In reply to Andi KleenQuoted 8 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Change
pmu.cto allow descriptions of events and add interfaces to add aliases at runtime from another file. To be used by jevents in the next patch.Acked-by Namhyung Kim <[email protected]>Signed-off-by Andi Kleen <[email protected]>
SNIP
tools/perf/util/pmu.c static char *format_alias_or(char *buf, int len, struct perf_pmu *pmu, · line 738
Quoted 9 lines
tools/perf/util/pmu.c +3
-0+return __perf_pmu__new_alias (list, name, dir,NULL , buf); +} +/* * Process all the sysfs attributes located under the directory * specified in 'dir' parameter. @@ -720,11 +737,51 @@ static char *format_alias_or(char *buf, int len, struct perf_pmu *pmu,return buf; }
changes below, the list command update should be separated patch
jirka
Quoted 125 lines
tools/perf/util/pmu.c +68
-16-static int cmp_string (const void *a,const void *b) +struct pair { +char *name; +char *desc; +}; + +static int cmp_pair (const void *a,const void *b) +{ +const struct pair *as = a; +const struct pair *bs = b; + +/* Put downloaded event list last */ +if (!!as->desc != !!bs->desc) +return !!as->desc - !!bs->desc; +return strcmp (as->name, bs->name); +} + +static void wordwrap (char *s,int start,int max,int corr) { -const char *const *as = a; -const char *const *bs = b; -return strcmp (*as, *bs); +int column = start; +int n; + +while (*s) { +int wlen =strcspn (s," \t" ); + +if (column + wlen >= max && column > start) { +printf ("\n%*s" , start,"" ); + column = start + corr; + } + n =printf ("%s%.*s" , column > start ?" " :"" , wlen, s); +if (n <=0 ) +break ; + s += wlen; + column += n; +while (isspace (*s)) + s++; + } +} + +static int get_columns (void ) +{ +/* +* Should ask the terminal with TIOCGWINSZ here, but we +* need the original fd before the pager. +*/ +return 79 ; }void print_pmu_events (const char *event_glob,bool name_only)@@ -734,21 +791,24 @@ void print_pmu_events(const char *event_glob, bool name_only)char buf[1024 ];int printed =0 ;int len, j; -char **aliases; +struct pair *aliases; +int numdesc =0 ; +int columns =get_columns (); pmu =NULL ; len =0 ;while ((pmu =perf_pmu__scan (pmu)) !=NULL )list_for_each_entry (alias, &pmu->aliases, list) len++; - aliases =malloc (sizeof (char *) * len); + aliases =malloc (sizeof (struct pair ) * len);if (!aliases)return ; pmu =NULL ; j =0 ;while ((pmu =perf_pmu__scan (pmu)) !=NULL )list_for_each_entry (alias, &pmu->aliases, list) { -char *name =format_alias (buf,sizeof (buf), pmu, alias); +char *name = alias->desc ? alias->name : +format_alias (buf,sizeof (buf), pmu, alias);bool is_cpu = !strcmp (pmu->name,"cpu" ); if (event_glob != NULL &&@@ -756,22 +816,31 @@ void print_pmu_events(const char *event_glob, bool name_only) (!is_cpu &&strglobmatch (alias->name, event_glob)))) continue; - aliases[j] = name; -if (is_cpu && !name_only) - aliases[j] =format_alias_or (buf,sizeof (buf), - pmu, alias); - aliases[j] =strdup (aliases[j]); + aliases[j].name = name; +if (is_cpu && !name_only && !alias->desc) + aliases[j].name =format_alias_or (buf, +sizeof (buf), + pmu, alias); + aliases[j].name =strdup (aliases[j].name); + aliases[j].desc = alias->desc; j++; } len = j; -qsort (aliases, len,sizeof (char *), cmp_string); +qsort (aliases, len,sizeof (struct pair ), cmp_pair);for (j =0 ; j < len; j++) {if (name_only) { -printf ("%s " , aliases[j]); +printf ("%s " , aliases[j].name);continue ; } -printf (" %-50s [Kernel PMU event]\n" , aliases[j]); -zfree (&aliases[j]); +if (aliases[j].desc) { +if (numdesc++ ==0 && printed) +printf ("\n" ); +printf (" %-50s [" , aliases[j].name); +wordwrap (aliases[j].desc,53 , columns,1 ); +printf ("]\n" ); + }else +printf (" %-50s [Kernel PMU event]\n" , aliases[j].name); +zfree (&aliases[j].name); printed++; }if (printed)Signature · 1 line
1.9.3
PATCH 3/9 perf, tools: Add support for reading JSON event files v3
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add a parser for Intel style JSON event files. This allows to use an Intel event list directly with perf. The Intel event lists can be quite large and are too big to store in unswappable kernel memory.
The parser code knows how to convert the JSON fields
to perf fields. The conversion code is straight forward. It knows (very little) Intel specific information, and can be easily extended to handle fields for other CPUs.
The parser code is partially shared with an independent parsing library, which is 2-clause BSD licenced. To avoid any conflicts I marked those files as BSD licenced too. As part of perf they become GPLv2.
The events are handled using the existing alias machinery.
We output the BriefDescription in perf list.
Right now the json file can be specified as an argument to perf stat/record/list. Followon patches will automate this.
JSON files look like this:
[
{
"EventCode": "0x00",
"UMask": "0x01",
"EventName": "INST_RETIRED.ANY",
"BriefDescription": "Instructions retired from execution.",
"PublicDescription": "Instructions retired from execution.",
"Counter": "Fixed counter 1",
"CounterHTOff": "Fixed counter 1",
"SampleAfterValue": "2000003",
"MSRIndex": "0",
"MSRValue": "0",
"TakenAlone": "0",
"CounterMask": "0",
"Invert": "0",
"AnyThread": "0",
"EdgeDetect": "0",
"PEBS": "0",
"PRECISE_STORE": "0",
"Errata": "null",
"Offcore": "0"},
v2: Address review feedback. Rename option to --event-files
v3: Add JSON example
--- tools/perf/Documentation/perf-list.txt | 6 + tools/perf/Documentation/perf-record.txt | 3 + tools/perf/Documentation/perf-stat.txt | 3 + tools/perf/Makefile.perf | 2 + tools/perf/builtin-list.c | 2 + tools/perf/builtin-record.c | 3 + tools/perf/builtin-stat.c | 2 + tools/perf/util/jevents.c | 250 +++++++++++++++++++++++++++++++ tools/perf/util/jevents.h | 3 + tools/perf/util/pmu.c | 14 ++ tools/perf/util/pmu.h | 2 + 11 files changed, 290 insertions(+) create mode 100644 tools/perf/util/jevents.c create mode 100644 tools/perf/util/jevents.h
tools/perf/Documentation/perf-list.txt +6 -0
@@ -15,6 +15,12 @@ DESCRIPTION This command displays the symbolic event types which can be selected in the various perf commands with the -e option. +OPTIONS +------- +--events-file=:: +Specify JSON event list file to use for parsing events. + + [[EVENT_MODIFIERS]] EVENT MODIFIERS ---------------
tools/perf/Documentation/perf-record.txt +3 -0
@@ -214,6 +214,9 @@ if combined with -a or -C options. After starting the program, wait msecs before measuring. This is useful to filter out the startup phase of the program, which is often very different. +--events-file=:: +Specify JSON event list file to use for parsing events. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/Documentation/perf-stat.txt +3 -0
@@ -142,6 +142,9 @@ filter out the startup phase of the program, which is often very different. Print statistics of transactional execution if supported. +--events-file=:: +Specify JSON event list file to use for parsing events. + EXAMPLES --------
tools/perf/Makefile.perf +2 -0
@@ -302,6 +302,7 @@ LIB_H += ui/ui.h LIB_H += util/data.h LIB_H += util/jsmn.h LIB_H += util/json.h +LIB_H += util/jevents.h LIB_OBJS += $(OUTPUT)util/abspath.o LIB_OBJS += $(OUTPUT)util/alias.o
@@ -377,6 +378,7 @@ LIB_OBJS += $(OUTPUT)util/srcline.o LIB_OBJS += $(OUTPUT)util/data.o LIB_OBJS += $(OUTPUT)util/jsmn.o LIB_OBJS += $(OUTPUT)util/json.o +LIB_OBJS += $(OUTPUT)util/jevents.o LIB_OBJS += $(OUTPUT)ui/setup.o LIB_OBJS += $(OUTPUT)ui/helpline.o
tools/perf/builtin-list.c +2 -0
@@ -20,6 +20,8 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused) {int i;const struct option list_options[] = { +OPT_STRING (0 ,"events-file" , &json_file,"json file" , +"Read event json file" ),OPT_END () };const char *const list_usage[] = {
tools/perf/builtin-record.c +3 -0
@@ -25,6 +25,7 @@#include "util/cpumap.h" #include "util/thread_map.h" #include "util/data.h" +#include "util/pmu.h" #include <unistd.h> #include <sched.h>
@@ -876,6 +877,8 @@ const struct option record_options[] = { "sample transaction flags (special events only)"),OPT_BOOLEAN (0 ,"per-thread" , &record.opts.target.per_thread,"use per-thread mmaps" ), +OPT_STRING (0 ,"events-file" , &json_file,"json file" , +"Read event json file" ), OPT_END() };
tools/perf/builtin-stat.c +2 -0
@@ -1668,6 +1668,8 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) "aggregate counts per physical processor core", AGGR_CORE),OPT_UINTEGER ('D' ,"delay" , &initial_delay,"ms to wait before starting measurement after program start" ), +OPT_STRING (0 ,"events-file" , &json_file,"json file" , +"Read event json file" ),OPT_END () }; const char * const stat_usage[] = {
tools/perf/util/jevents.c new file +250 -0
250 lines not on this page · raw message
tools/perf/util/jevents.h new file +3 -0
@@ -0,0 +1,3 @@ +int json_events (const char *fn, +int (*func)(void *data,char *name,char *event,char *desc), +void *data);
tools/perf/util/pmu.c +14 -0
@@ -9,6 +9,9 @@#include "pmu.h" #include "parse-events.h" #include "cpumap.h" +#include "jevents.h" + +const char *json_file;#define UNIT_MAX_LEN 31/* max length for event unit name */
@@ -273,6 +276,14 @@ static int pmu_aliases_parse(char *dir, struct list_head *head)return ret; } +static int add_alias (void *data,char *name,char *event,char *desc) +{ +struct list_head *head = (struct list_head *)data; + +__perf_pmu__new_alias (head, name,NULL , desc, event); +return 0 ; +} +/* * Reading the pmu event aliases definition, which should be located at: * /sys/bus/event_source/devices/<dev>/events as sysfs group attributes.
@@ -422,6 +433,9 @@ static struct perf_pmu *pmu_lookup(const char *name)if (pmu_aliases (name, &aliases))return NULL ; +if (!strcmp (name,"cpu" ) && json_file) +json_events (json_file, add_alias, &aliases); +if (pmu_type (name, &type))return NULL ;
tools/perf/util/pmu.h +2 -0
@@ -46,4 +46,6 @@ void print_pmu_events(const char *event_glob, bool name_only);bool pmu_have_event (const char *pname,const char *name);int perf_pmu__test (void ); + +extern const char *json_file;#endif /* __PMU_H */
Signature · 1 line
1.9.3
Re: [PATCH 3/9] perf, tools: Add support for reading JSON event files v3
Quoted 7 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Add a parser for Intel style JSON event files. This allows to use an Intel event list directly with perf. The Intel event lists can be quite large and are too big to store in unswappable kernel memory.
Quoted 8 lines
The parser code knows how to convert the JSON fields
to perf fields. The conversion code is straight forward. It knows (very little) Intel specific information, and can be easily extended to handle fields for other CPUs.The parser code is partially shared with an independent parsing library, which is 2-clause BSD licenced. To avoid any conflicts I marked those files as BSD licenced too. As part of perf they become GPLv2.
this change breaks compilation, which is probably fixed later on in patchset
CC util/jevents.outil/jevents.c: In function ‘lookup_msr’:util/jevents.c:140:2: error: unknown type name ‘bool’
static bool warned;
^util/jevents.c:148:12: error: ‘true’ undeclared (first use in this function)
warned = true;
^util/jevents.c:148:12: note: each undeclared identifier is reported only once for each function it appears in util/jevents.c:149:3: error: implicit declaration of function ‘pr_err’ [-Werror=implicit-function-declaration]
pr_err("Unknown MSR in event file %.*s\n",
^util/jevents.c:149:3: error: nested extern declaration of ‘pr_err’ [-Werror=nested-externs]
cc1: all warnings being treated as errors
make[1]: *** [util/jevents.o] Error 1
make: *** [all] Error 2
jirka
Re: [PATCH 3/9] perf, tools: Add support for reading JSON event files v3
In reply to Andi KleenQuoted 1 line · Andi Kleen
From: Andi Kleen <[email protected]>
SNIP
tools/perf/util/jevents.h
Quoted 7 lines
(quoted diff) +1
-0+}tools/perf/util/jevents.h new file +0
-0@@ -0,0 +1,3 @@
missing standard '#ifndef ...' macros
tools/perf/util/jevents.h line 3
Quoted 3 lines
tools/perf/util/jevents.h new file +3
-0+int json_events (const char *fn, +int (*func)(void *data,char *name,char *event,char *desc), +void *data);
jirka
Re: [PATCH 3/9] perf, tools: Add support for reading JSON event files v3
In reply to Andi KleenIm confused, your branch shows v4 in patch Subject, while this patch has v3... ?
jirka
Quoted 16 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Add a parser for Intel style JSON event files. This allows to use an Intel event list directly with perf. The Intel event lists can be quite large and are too big to store in unswappable kernel memory.
The parser code knows how to convert the JSON fields
to perf fields. The conversion code is straight forward. It knows (very little) Intel specific information, and can be easily extended to handle fields for other CPUs.The parser code is partially shared with an independent parsing library, which is 2-clause BSD licenced. To avoid any conflicts I marked those files as BSD licenced too. As part of perf they become GPLv2.
SNIP
Re: [PATCH 3/9] perf, tools: Add support for reading JSON event files v3
Quoted 2 lines · Jiri Olsa
Im confused, your branch shows v4 in patch Subject, while this patch has v3... ?
Patchkits and patches both have their own numeric
version spaces.
-Andi
Re: [PATCH 3/9] perf, tools: Add support for reading JSON event files v3
Quoted 6 lines · Andi Kleen
Quoted 2 lines · Jiri Olsa
Im confused, your branch shows v4 in patch Subject, while this patch has v3... ?
Patchkits and patches both have their own numeric
version spaces.
yes, but this specific patch has higher (different) version in the branch, that patch you sent
jirka
Re: [PATCH 3/9] perf, tools: Add support for reading JSON event files v3
Quoted 10 lines · Jiri Olsa
Quoted 6 lines · Andi Kleen
Quoted 2 lines · Jiri Olsa
Im confused, your branch shows v4 in patch Subject, while this patch has v3... ?
Patchkits and patches both have their own numeric
version spaces.yes, but this specific patch has higher (different) version in the branch, that patch you sent
Yes the branch contains an unposted version.
Only change was just the manpage changes that were requested during review.
-Andi
Re: [PATCH 3/9] perf, tools: Add support for reading JSON event files v3
In reply to Andi KleenQuoted 15 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Add a parser for Intel style JSON event files. This allows to use an Intel event list directly with perf. The Intel event lists can be quite large and are too big to store in unswappable kernel memory.
The parser code knows how to convert the JSON fields
to perf fields. The conversion code is straight forward. It knows (very little) Intel specific information, and can be easily extended to handle fields for other CPUs.The parser code is partially shared with an independent parsing library, which is 2-clause BSD licenced. To avoid any conflicts I marked those files as BSD licenced too. As part of perf they become GPLv2.
SNIP
Quoted 11 lines
tools/perf/util/jevents.c new file +11
-0+ +#include <stdio.h> +#include <stdlib.h> +#include <errno.h> +#include <string.h> +#include <ctype.h> +#include "jsmn.h" +#include "json.h" +#include "jevents.h" + +static void addfield(char *map, char **dst, const char *sep,
tools/perf/util/jevents.c line 50
Quoted 10 lines
tools/perf/util/jevents.c new file +10
-0+ const char *a, jsmntok_t *bt) +{ + unsigned len = strlen(a) + 1 + strlen(sep); +int olen = *dst ?strlen (*dst) :0 ; +int blen = bt ?json_len (bt) :0 ; +char *out; + + out =realloc (*dst, len + olen + blen); +if (!out) +return ;
what happens if we fail to realloc?
jirka
Re: [PATCH 3/9] perf, tools: Add support for reading JSON event files v3
Quoted 1 line
what happens if we fail to realloc?
Then the field doesn't get added. No problem.
-Andi
PATCH 4/9 perf, tools: Automatically look for event file name for cpu v3
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
When no JSON event file is specified automatically look for a suitable file in ~/.cache/pmu-events. A "perf download" can automatically add files there for the current CPUs.
This does not include the actual event files with perf, but they can be automatically downloaded instead (implemented in the next patch)
This has the advantage that the events can be always uptodate, because they are freshly downloaded. In oprofile we always had problems with out of date or incomplete events files.
The event file format is per architecture, but can be extended for other architectures.
v2: Supports XDG_CACHE_HOME and defaults to ~/.cache/pmu-events
v3: Minor updates and handle EVENTMAP.
--- tools/perf/arch/x86/Makefile | 1 + tools/perf/arch/x86/util/cpustr.c | 34 ++++++++++++++++++++++++++++++++ tools/perf/util/jevents.c | 41 +++++++++++++++++++++++++++++++++++++++ tools/perf/util/jevents.h | 1 + tools/perf/util/pmu.c | 2 +- 5 files changed, 78 insertions(+), 1 deletion(-) create mode 100644 tools/perf/arch/x86/util/cpustr.c
tools/perf/arch/x86/Makefile +1 -0
@@ -14,4 +14,5 @@ LIB_OBJS += $(OUTPUT)arch/$(ARCH)/tests/dwarf-unwind.o endif LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/header.o LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/tsc.o +LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/cpustr.o LIB_H += arch/$(ARCH)/util/tsc.h
tools/perf/arch/x86/util/cpustr.c new file +34 -0
@@ -0,0 +1,34 @@ +#include <stdio.h> +#include <stdlib.h> +#include "../../util/jevents.h" + +char *get_cpu_str (void ) +{ +char *line =NULL ; +size_t llen =0 ; +int found =0 , n; +char vendor[30 ]; +int model, fam; +char *res =NULL ; +FILE *f =fopen ("/proc/cpuinfo" ,"r" ); + +if (!f) +return NULL ; +while (getline (&line, &llen, f) >0 ) { +if (sscanf (line,"vendor_id : %30s" , vendor) ==1 ) + found++; +else if (sscanf (line,"model : %d" , &model) ==1 ) + found++; +else if (sscanf (line,"cpu family : %d" , &fam) ==1 ) + found++; +if (found ==3 ) { + n =asprintf (&res,"%s-%d-%X-core" , vendor, fam, model); +if (n <0 ) + res =NULL ; +break ; + } + } +free (line); +fclose (f); +return res; +}
tools/perf/util/jevents.c +41 -0
57 lines not on this page · raw message
tools/perf/util/jevents.h +1 -0
@@ -1,3 +1,4 @@int json_events (const char *fn,int (*func)(void *data,char *name,char *event,char *desc),void *data); +char *get_cpu_str (void );
tools/perf/util/pmu.c +1 -1
@@ -433,7 +433,7 @@ static struct perf_pmu *pmu_lookup(const char *name)if (pmu_aliases (name, &aliases))return NULL ; -if (!strcmp (name,"cpu" ) && json_file) +if (!strcmp (name,"cpu" ))json_events (json_file, add_alias, &aliases);if (pmu_type (name, &type))
Signature · 1 line
1.9.3
Re: [PATCH 4/9] perf, tools: Automatically look for event file name for cpu v3
Quoted 1 line · Andi Kleen
From: Andi Kleen <[email protected]>
SNIP
Quoted 28 lines
new file mode 100644
index 0000000..e1cd76ctools/perf/arch/x86/util/cpustr.c new file +23
-0@@ -0,0 +1,34 @@ +#include <stdio.h> +#include <stdlib.h> +#include "../../util/jevents.h" + +char *get_cpu_str (void ) +{ +char *line =NULL ; +size_t llen =0 ; +int found =0 , n; +char vendor[30 ]; +int model, fam; +char *res =NULL ; +FILE *f =fopen ("/proc/cpuinfo" ,"r" ); + +if (!f) +return NULL ; +while (getline (&line, &llen, f) >0 ) { +if (sscanf (line,"vendor_id : %30s" , vendor) ==1 ) + found++; +else if (sscanf (line,"model : %d" , &model) ==1 ) + found++; +else if (sscanf (line,"cpu family : %d" , &fam) ==1 ) + found++;
tools/perf/arch/x86/util/cpustr.c line 33
Quoted 10 lines
tools/perf/arch/x86/util/cpustr.c new file +10
-0+if (found ==3 ) { + n =asprintf (&res,"%s-%d-%X-core" , vendor, fam, model); +if (n <0 ) + res =NULL ; +break ; + } + } +free (line); +fclose (f); +return res;
we already have arch get_cpuid, which seems to do same thing via cpuid it would be nice to share those if possible
jirka
PATCH 5/9 perf, tools: Add perf download to download event files v4
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add a downloader to automatically download the right files from a download site.
This is implemented as a script calling wget, similar to perf archive. The perf driver automatically calls the right binary. The downloader is extensible, but currently only implements an Intel event download. It would be straightforward to add other sites too for other vendors.
The downloaded event files are put into ~/.cache/pmu-events, where the builtin event parser in util/* can find them automatically.
v2: Use ~/.cache
v3: Check for wget. Some cleanups.
v4: Improve manpage.
--- tools/perf/Documentation/perf-download.txt | 31 ++++++++++++++++ tools/perf/Documentation/perf-list.txt | 12 ++++++- tools/perf/Makefile.perf | 5 ++- tools/perf/perf-download.sh | 57 ++++++++++++++++++++++++++++++ 4 files changed, 103 insertions(+), 2 deletions(-) create mode 100644 tools/perf/Documentation/perf-download.txt create mode 100755 tools/perf/perf-download.sh
tools/perf/Documentation/perf-download.txt new file +31 -0
@@ -0,0 +1,31 @@ +perf-download(1) +=============== + +NAME +---- +perf-download - Download event files for current CPU. + +SYNOPSIS +-------- +[verse] +'perf download' [vendor-family-model] + +DESCRIPTION +----------- +This command automatically downloads the event list for the current CPU and +stores them in $XDG_CACHE_HOME/pmu-events (or $HOME/.cache/pmu-events). +The other tools automatically look for them there. The CPU can be also +specified at the command line. + +The downloading is done using http through wget, which needs +to be installed. When behind a firewall the proxies +may also need to be set up using "export https_proxy=...." + +The user should regularly call this to download updated event lists +for the current CPU. + +Note the downloaded files are stored per user, so if perf is +used as both normal user and with sudo the event files may +also need to be moved to root's home directory with +sudo mkdir /root/.cache ; sud cp -r ~/.cache/pmu-events /root/.cache +after downloading.
tools/perf/Documentation/perf-list.txt +11 -1
@@ -61,6 +61,16 @@ Sampling). Examples to use IBS: perf record -a -e r076:p ... # same as -e cpu-cycles:p perf record -a -e r0C1:p ... # use ibs op counting micro-ops +PER CPU EVENT LISTS +------------------- + +For some CPUs (particularly modern Intel CPUs) "perf download" can +download additional CPU specific event definitions, which then +become visible in perf list and available in the other perf tools. + +This obsoletes the raw event description method described below +for most cases. + RAW HARDWARE EVENT DESCRIPTOR ----------------------------- Even when an event is not available in a symbolic form within perf right now,
@@ -123,6 +133,6 @@ types specified. SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-top[1], -linkperf:perf-record[1], +linkperf:perf-record[1], linkperf:perf-download[1], http://www.intel.com/Assets/PDF/manual/253669.pdf[Intel® 64 and IA-32 Architectures Software Developer's Manual Volume 3B: System Programming Guide], http://support.amd.com/us/Processor_TechDocs/24593_APM_v2.pdf[AMD64 Architecture Programmer’s Manual Volume 2: System Programming]
tools/perf/Makefile.perf +4 -1
@@ -126,6 +126,7 @@ PYRF_OBJS = SCRIPT_SH = SCRIPT_SH += perf-archive.sh +SCRIPT_SH += perf-download.sh grep-libs = $(filter -l%,$(1)) strip-libs = $(filter-out -l%,$(1))
@@ -877,6 +878,8 @@ install-bin: all install-gtk $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)' $(call QUIET_INSTALL, perf-archive) \ $(INSTALL) $(OUTPUT)perf-archive -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)' + $(call QUIET_INSTALL, perf-download) \ + $(INSTALL) $(OUTPUT)perf-download -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)' ifndef NO_LIBPERL $(call QUIET_INSTALL, perl-scripts) \ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl/Perf-Trace-Util/lib/Perf/Trace'; \
@@ -922,7 +925,7 @@ config-clean: @$(MAKE) -C config/feature-checks clean >/dev/null clean: $(LIBTRACEEVENT)-clean $(LIBAPIKFS)-clean config-clean - $(call QUIET_CLEAN, core-objs) $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf.o $(LANG_BINDINGS) $(GTK_OBJS) + $(call QUIET_CLEAN, core-objs) $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)/perf-download $(OUTPUT)perf.o $(LANG_BINDINGS) $(GTK_OBJS) $(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf $(call QUIET_CLEAN, core-gen) $(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope* $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)PERF-CFLAGS $(OUTPUT)PERF-FEATURES $(OUTPUT)util/*-bison* $(OUTPUT)util/*-flex* $(QUIET_SUBDIR0)Documentation $(QUIET_SUBDIR1) clean
tools/perf/perf-download.sh new file +57 -0
57 lines not on this page · raw message
Signature · 1 line
1.9.3
Re: [PATCH 5/9] perf, tools: Add perf download to download event files v4
Quoted 13 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Add a downloader to automatically download the right files from a download site.
This is implemented as a script calling wget, similar to perf archive. The perf driver automatically calls the right binary. The downloader is extensible, but currently only implements an Intel event download. It would be straightforward to add other sites too for other vendors.
The downloaded event files are put into ~/.cache/pmu-events, where the builtin event parser in util/* can find them automatically.
SNIP
Quoted 7 lines
tools/perf/Makefile.perf +0
-0$(QUIET_SUBDIR0)Documentation $(QUIET_SUBDIR1) cleantools/perf/perf-download.sh new file +0
-0
tools/perf/perf-download.sh line 9
Quoted 9 lines
tools/perf/perf-download.sh new file +9
-0@@ -0,0 +1,57 @@ +#!/bin/bash +# download event files for current cpu for perf + +WGETOPT=${WGETOPT:---no-verbose --timeout 5} + +set -e + +if ! type wget > /dev/null ; then + echo "please install wget"
I guess the other alternative is curl.. could u check for this one as well?
jirka
Re: [PATCH 5/9] perf, tools: Add perf download to download event files v4
Quoted 1 line
I guess the other alternative is curl.. could u check for this one as well?
I looked at this earlier, but it made a complete mess of the download script to handle both. So right now I stayed with wget only.
-Andi
PATCH 6/9 perf, tools: Allow events with dot
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
The Intel events use a dot to separate event name and unit mask. Allow dot in names in the scanner, and remove special handling of dot as EOF. Also remove the hack in jevents to replace dot with underscore. This way dotted events can be specified directly by the user.
I'm not fully sure this change to the scanner is correct (what was the dot special case good for?), but I haven't found anything that breaks with it so far at least.
--- tools/perf/util/jevents.c | 9 +-------- tools/perf/util/parse-events.l | 3 +-- 2 files changed, 2 insertions(+), 10 deletions(-)
tools/perf/util/jevents.c +1 -8
@@ -100,15 +100,8 @@ static void addfield(char *map, char **dst, const char *sep,static void fixname (char *s) { -for (; *s; s++) { +for (; *s; s++) *s =tolower (*s); -/* -* Remove '.' for now, until the parser -* can deal with it. -*/ -if (*s =='.' ) - *s ='_' ; - } }static void fixdesc (char *s)
tools/perf/util/parse-events.l +1 -2
@@ -81,7 +81,7 @@ num_dec [0-9]+ num_hex 0x[a-fA-F0-9]+ num_raw_hex [a-fA-F0-9]+ name [a-zA-Z_*?][a-zA-Z0-9_*?]* -name_minus [a-zA-Z_*?][a-zA-Z0-9\-_*?]* +name_minus [a-zA-Z_*?][a-zA-Z0-9\-_*?.]* /* If you add a modifier you need to update check_modifier() */ modifier_event [ukhpGHSD]+ modifier_bp [rwx]{1,3}
@@ -119,7 +119,6 @@ modifier_bp [rwx]{1,3} return PE_EVENT_NAME; } -. | <<EOF>> { BEGIN(INITIAL); yyless(0); }
Signature · 1 line
1.9.3
Re: [PATCH 6/9] perf, tools: Allow events with dot
Quoted 28 lines · Andi Kleen
From: Andi Kleen <[email protected]>
The Intel events use a dot to separate event name and unit mask. Allow dot in names in the scanner, and remove special handling of dot as EOF. Also remove the hack in jevents to replace dot with underscore. This way dotted events can be specified directly by the user.
I'm not fully sure this change to the scanner is correct (what was the dot special case good for?), but I haven't found anything that breaks with it so far at least.
Acked-by Namhyung Kim <[email protected]>Signed-off-by Andi Kleen <[email protected]>--- tools/perf/util/jevents.c | 9 +-------- tools/perf/util/parse-events.l | 3 +-- 2 files changed, 2 insertions(+), 10 deletions(-)tools/perf/util/jevents.c +0
-1@@ -100,15 +100,8 @@ static void addfield(char *map, char **dst, const char *sep,static void fixname (char *s) { -for (; *s; s++) {
tools/perf/util/jevents.c static void addfield(char *map, char **dst, const char *sep, · line 105
Quoted 10 lines
tools/perf/util/jevents.c +1
-7+for (; *s; s++) *s =tolower (*s); -/* -* Remove '.' for now, until the parser -* can deal with it. -*/ -if (*s =='.' ) - *s ='_' ; - } }
please submit change below as separated patch
before above 'fixname' is introduced
jirka
Quoted 26 lines
tools/perf/util/jevents.c +0
-0static void fixdesc (char *s)tools/perf/util/parse-events.l +1
-2@@ -81,7 +81,7 @@ num_dec [0-9]+ num_hex 0x[a-fA-F0-9]+ num_raw_hex [a-fA-F0-9]+ name [a-zA-Z_*?][a-zA-Z0-9_*?]* -name_minus [a-zA-Z_*?][a-zA-Z0-9\-_*?]* +name_minus [a-zA-Z_*?][a-zA-Z0-9\-_*?.]* /* If you add a modifier you need to update check_modifier() */ modifier_event [ukhpGHSD]+ modifier_bp [rwx]{1,3}@@ -119,7 +119,6 @@ modifier_bp [rwx]{1,3} return PE_EVENT_NAME; } -. | <<EOF>> { BEGIN(INITIAL); yyless(0); }Signature · 1 line
1.9.3
PATCH 7/9 perf, tools: Query terminal width and use in perf list
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Automatically adapt the now wider and word wrapped perf list output to wider terminals. This requires querying the terminal before the auto pager takes over, and exporting this information from the pager subsystem.
--- tools/perf/util/cache.h | 1 + tools/perf/util/pager.c | 15 +++++++++++++++ tools/perf/util/pmu.c | 12 ++---------- 3 files changed, 18 insertions(+), 10 deletions(-)
tools/perf/util/cache.h +1 -0
@@ -31,6 +31,7 @@ extern void setup_pager(void);extern const char *pager_program;extern int pager_in_use (void );extern int pager_use_color; +int pager_get_columns (void );char *alias_lookup (const char *alias);int split_cmdline (char *cmdline,const char ***argv);
tools/perf/util/pager.c +15 -0
@@ -1,6 +1,7 @@#include "cache.h" #include "run-command.h" #include "sigchain.h" +#include <sys/ioctl.h> /* * This is split up from the rest of git so that we can do
@@ -8,6 +9,7 @@*/ static int spawned_pager; +static int pager_columns;static void pager_preexec (void ) {
@@ -47,9 +49,12 @@ static void wait_for_pager_signal(int signo)void setup_pager (void ) {const char *pager =getenv ("PERF_PAGER" ); +struct winsize sz;if (!isatty (1 ))return ; +if (ioctl (1 ,TIOCGWINSZ , &sz) ==0 ) + pager_columns = sz.ws_col;if (!pager) {if (!pager_program)perf_config (perf_default_config,NULL );
@@ -98,3 +103,13 @@ int pager_in_use(void) env =getenv ("PERF_PAGER_IN_USE" );return env ?perf_config_bool ("PERF_PAGER_IN_USE" , env) :0 ; } + +int pager_get_columns (void ) +{ +char *s; + + s =getenv ("COLUMNS" ); +if (s) +return atoi (s); +return (pager_columns ? pager_columns :80 ) -2 ; +}
tools/perf/util/pmu.c +2 -10
@@ -9,6 +9,7 @@#include "pmu.h" #include "parse-events.h" #include "cpumap.h" +#include "cache.h" #include "jevents.h" const char *json_file;
@@ -789,15 +790,6 @@ static void wordwrap(char *s, int start, int max, int corr) } } -static int get_columns (void ) -{ -/* -* Should ask the terminal with TIOCGWINSZ here, but we -* need the original fd before the pager. -*/ -return 79 ; -} -void print_pmu_events (const char *event_glob,bool name_only) {struct perf_pmu *pmu;
@@ -807,7 +799,7 @@ void print_pmu_events(const char *event_glob, bool name_only)int len, j;struct pair *aliases;int numdesc =0 ; -int columns =get_columns (); +int columns =pager_get_columns (); pmu =NULL ; len =0 ;
Signature · 1 line
1.9.3
PATCH 8/9 perf, tools, test: Add test case for alias and JSON parsing v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add a simple test case to perf test that runs perf download and parses all the available events, including json events.
This needs adding an all event iterator to pmu.c
v2: Rename identifiers
--- tools/perf/Makefile.perf | 1 + tools/perf/tests/aliases.c | 58 +++++++++++++++++++++++++++++++++++++++++ tools/perf/tests/builtin-test.c | 4 +++ tools/perf/tests/tests.h | 1 + tools/perf/util/pmu.c | 18 +++++++++++++ tools/perf/util/pmu.h | 2 ++ 6 files changed, 84 insertions(+) create mode 100644 tools/perf/tests/aliases.c
tools/perf/Makefile.perf +1 -0
@@ -419,6 +419,7 @@ endif LIB_OBJS += $(OUTPUT)tests/code-reading.o LIB_OBJS += $(OUTPUT)tests/sample-parsing.o LIB_OBJS += $(OUTPUT)tests/parse-no-sample-id-all.o +LIB_OBJS += $(OUTPUT)tests/aliases.o ifndef NO_DWARF_UNWIND ifeq ($(ARCH),$(filter $(ARCH),x86 arm)) LIB_OBJS += $(OUTPUT)tests/dwarf-unwind.o
tools/perf/tests/aliases.c new file +58 -0
58 lines not on this page · raw message
tools/perf/tests/builtin-test.c +4 -0
@@ -154,6 +154,10 @@ static struct test { .func = test__hists_cumulate, }, { + .desc = "Test parsing JSON aliases", + .func = test__aliases, + }, + { .func = NULL, }, };
tools/perf/tests/tests.h +1 -0
@@ -48,6 +48,7 @@ int test__mmap_thread_lookup(void);int test__thread_mg_share (void );int test__hists_output (void );int test__hists_cumulate (void ); +int test__aliases (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
tools/perf/util/pmu.c +18 -0
@@ -869,3 +869,21 @@ bool pmu_have_event(const char *pname, const char *name) }return false; } + +int pmu_iterate_events (int (*func)(const char *name)) +{ +int ret =0 ; +struct perf_pmu *pmu; +struct perf_pmu_alias *alias; + +perf_pmu__find ("cpu" );/* Load PMUs */ + pmu =NULL ; +while ((pmu =perf_pmu__scan (pmu)) !=NULL ) { +list_for_each_entry (alias, &pmu->aliases, list) { + ret =func (alias->name); +if (ret !=0 ) +break ; + } + } +return ret; +}
tools/perf/util/pmu.h +2 -0
@@ -47,5 +47,7 @@ bool pmu_have_event(const char *pname, const char *name);int perf_pmu__test (void ); +int pmu_iterate_events (int (*func)(const char *name)); +extern const char *json_file;#endif /* __PMU_H */
Signature · 1 line
1.9.3
Re: [PATCH 8/9] perf, tools, test: Add test case for alias and JSON parsing v2
Quoted 6 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Add a simple test case to perf test that runs perf download and parses all the available events, including json events.
This needs adding an all event iterator to
pmu.c
I still get this error display
33: Test parsing JSON aliases :invalid or unsupported event: 'clockticks'
invalid or unsupported event: 'clockticks'
Parsed 521 events : Ok
I guess you could iterate just cpu PMU
jirka
Re: [PATCH 8/9] perf, tools, test: Add test case for alias and JSON parsing v2
In reply to Andi KleenQuoted 7 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Add a simple test case to perf test that runs perf download and parses all the available events, including json events.
This needs adding an all event iterator to
pmu.c
SNIP
Quoted 23 lines
tools/perf/util/pmu.c +18
-0+ +int pmu_iterate_events (int (*func)(const char *name)) +{ +int ret =0 ; +struct perf_pmu *pmu; +struct perf_pmu_alias *alias; + +perf_pmu__find ("cpu" );/* Load PMUs */ + pmu =NULL ; +while ((pmu =perf_pmu__scan (pmu)) !=NULL ) { +list_for_each_entry (alias, &pmu->aliases, list) { + ret =func (alias->name); +if (ret !=0 ) +break ; + } + } +return ret; +}tools/perf/util/pmu.h +0
-0
tools/perf/util/pmu.h bool pmu_have_event(const char *pname, const char *name); · line 51
Quoted 5 lines
tools/perf/util/pmu.h +2
-0@@ -47,5 +47,7 @@ bool pmu_have_event(const char *pname, const char *name);int perf_pmu__test (void ); +int pmu_iterate_events (int (*func)(const char *name)); +
this new pmu interface needs to go to separated patchset
jirka
PATCH 9/9 perf, tools: Add a --quiet flag to perf list
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add a --quiet flag to perf list to not print the event descriptions that were earlier added for JSON events. This may be useful to get a less crowded listing.
It's still default to print descriptions as that is the more useful default for most users.
Before:
% perf list
...
baclears.any [Counts the total number when the front end is
resteered, mainly when the BPU cannot provide a correct prediction and this is corrected by other branch handling mechanisms at the front end]
br_inst_exec.all_branches [Speculative and retired branches]
After:
% perf list --quiet
...
baclears.any [Kernel PMU event]
br_inst_exec.all_branches [Kernel PMU event]
--- tools/perf/builtin-list.c | 14 +++++++++----- tools/perf/util/parse-events.c | 4 ++-- tools/perf/util/parse-events.h | 2 +- tools/perf/util/pmu.c | 4 ++-- tools/perf/util/pmu.h | 2 +- 5 files changed, 15 insertions(+), 11 deletions(-)
tools/perf/builtin-list.c +9 -5
@@ -16,16 +16,20 @@#include "util/pmu.h" #include "util/parse-options.h" +static bool quiet_flag; +int cmd_list (int argc,const char **argv,const char *prefix__maybe_unused ) {int i;const struct option list_options[] = {OPT_STRING (0 ,"events-file" , &json_file,"json file" ,"Read event json file" ), +OPT_BOOLEAN ('q' ,"quiet" , &quiet_flag, +"Don't print extra event descriptions" ),OPT_END () };const char *const list_usage[] = { -"perf list [hw|sw|cache|tracepoint|pmu|event_glob]" , +"perf list [--events-file FILE] [--quiet] [hw|sw|cache|tracepoint|pmu|event_glob]" ,NULL };
@@ -35,7 +39,7 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)setup_pager ();if (argc ==0 ) { -print_events (NULL , false); +print_events (NULL , false, quiet_flag);return 0 ; }
@@ -54,15 +58,15 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused) strcmp(argv[i], "hwcache") == 0)print_hwcache_events (NULL , false); else if (strcmp(argv[i], "pmu") == 0) - print_pmu_events(NULL, false); + print_pmu_events(NULL, false, quiet_flag); else if (strcmp(argv[i], "--raw-dump") == 0) - print_events(NULL , true); + print_events(NULL , true, quiet_flag); else {char *sep =strchr (argv[i],':' ), *s;int sep_idx;if (sep ==NULL ) { -print_events (argv[i], false); +print_events (argv[i], false, quiet_flag);continue ; } sep_idx = sep - argv[i];
tools/perf/util/parse-events.c +2 -2
@@ -1231,7 +1231,7 @@ static void print_symbol_events(const char *event_glob, unsigned type,/* * Print the help text for the event symbols: */ -void print_events (const char *event_glob,bool name_only) +void print_events (const char *event_glob,bool name_only,bool quiet) {if (!name_only) {printf ("\n" );
@@ -1246,7 +1246,7 @@ void print_events(const char *event_glob, bool name_only)print_hwcache_events (event_glob, name_only); -print_pmu_events (event_glob, name_only); +print_pmu_events (event_glob, name_only, quiet);if (event_glob !=NULL )return ;
tools/perf/util/parse-events.h +1 -1
@@ -100,7 +100,7 @@ void parse_events_update_lists(struct list_head *list_event, struct list_head *list_all);void parse_events_error (void *data,void *scanner,char const *msg); -void print_events (const char *event_glob,bool name_only); +void print_events (const char *event_glob,bool name_only,bool quiet);void print_events_type (u8 type);void print_tracepoint_events (const char *subsys_glob,const char *event_glob,bool name_only);
tools/perf/util/pmu.c +2 -2
@@ -790,7 +790,7 @@ static void wordwrap(char *s, int start, int max, int corr) } } -void print_pmu_events (const char *event_glob,bool name_only) +void print_pmu_events (const char *event_glob,bool name_only,bool quiet_flag) {struct perf_pmu *pmu;struct perf_pmu_alias *alias;
@@ -838,7 +838,7 @@ void print_pmu_events(const char *event_glob, bool name_only)printf ("%s " , aliases[j].name);continue ; } -if (aliases[j].desc) { +if (aliases[j].desc && !quiet_flag) {if (numdesc++ ==0 && printed)printf ("\n" );printf (" %-50s [" , aliases[j].name);
tools/perf/util/pmu.h +1 -1
@@ -42,7 +42,7 @@ int perf_pmu__format_parse(char *dir, struct list_head *head);struct perf_pmu *perf_pmu__scan (struct perf_pmu *pmu); -void print_pmu_events (const char *event_glob,bool name_only); +void print_pmu_events (const char *event_glob,bool name_only,bool quiet);bool pmu_have_event (const char *pname,const char *name);int perf_pmu__test (void );
Signature · 1 line
1.9.3
Andi Kleen [[email protected]] wrote:
| Should be ready for merge now. Please consider.
Overall I think it is a cool feature. I was able to run some simple tests on Power8 (by explicitly specifying the JSON file). Have a couple of questions below.
| | [v2: Review feedback addressed and some minor improvements] | [v3: More review feedback addressed and handle test failures better. | Ported to latest tip/core.] | [v4: Addressed Namhyung's feedback] | [v5: Rebase to latest tree. Minor description update.] | [v6: Rebase. Add acked by from Namhyung and address feedback. Some minor | fixes. Should be good to go now I hope. The period patch was dropped, | as that is already handled. I added an extra patch for a --quiet argument | for perf list] | [v7: Just rebase to latest tip/core. Should be ready to merge.] | | perf has high level events which are useful in many cases. However | there are some tuning situations where low level events in the CPU | are needed. Traditionally this required specifying the event in | raw form (very awkward) or using non standard frontends | like ocperf or patching in libpfm. | | Intel CPUs can have very large event files (Haswell has ~336 core events, | much more if you add uncore or all the offcore combinations), which is too | large to describe through the kernel interface. It would require tying up | significant amounts of unswappable memory for this. | | oprofile always had separate event list files that were maintained by | the CPU vendors. The oprofile events were shipped with the tool. | The Intel events get updated regularly, for example to add references | to the specification updates or add new events. | | Unfortunately oprofile usually did not keep up with these updates, | so the events in oprofile were often out of date. In addition | it ties up quite a bit of disk space, mostly for CPUs you don't have. | | This patch kit implements another mechanism that avoids these problems. | Intel releases the event lists for CPUs in a standardized JSON format | on a download server. | | I implemented an automatic downloader to get the event file for the | current CPU. The events are stored in ~/.cache/pmu-events. | Then perf adds a parser that converts the JSON format into perf event | aliases, which then can be used directly as any other perf event. | | The parsing is done using a simple existing JSON library. | | The events are still abstracted for perf, but the abstraction mechanism is | through the downloaded file instead of through the kernel. | | The JSON format and perf parser has some minor Intelisms, but they | are simple and small and optional. It's easy to extend, so it would be | possible to use it for other CPUs too, add different pmu attributes, and | add new download sites to the downloader tool.
Is there a minimal set of JSON entries an architecture would need ?
I tried the following on Power
[
{
"EventCode": "2",
"EventName": "PM_INST_CMPL",
"BriefDescription": "Instructions completed",
"PublicDescription": "Number of PPC instructions finished",
},
{
"EventCode": "0x1E",
"EventName": "PM_CYC",
"BriefDescription": "Cycles completed",
"PublicDescription": "Number of PPC cycles finished",
}
]
/tmp/perf record --events-file=/tmp/power8.json -e PM_INST_CMPL sleep 1works, but for some TBD reason,
/tmp/perf list --events-file=/tmp/power8.json doesn't list PM_INST_CMPL.
Another observation was that the order of --events-file and -e is significant. Maybe worth a note in the man page.
| | Currently only core events are supported, uncore may come at a later | point. No kernel changes, all code in perf user tools only. | | Some of the parser files are partially shared with separate event parser | library and are thus 2-clause BSD licensed. | | Patches also available from | git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc perf/json | | Example output: | | % perf download | Downloading models file | Downloading readme.txt | 2014-03-05 10:39:33 URL:https://download.01.org/perfmon/readme.txt [10320/10320] -> "readme.txt" [1] | 2014-03-05 10:39:34 URL:https://download.01.org/perfmon/mapfile.csv [1207/1207] -> "mapfile.csv" [1] | Downloading events file | % perf list | ... | br_inst_exec.all_branches [Speculative and retired | branches] | br_inst_exec.all_conditional [Speculative and retired | macro-conditional | branches] | br_inst_exec.all_direct_jmp [Speculative and retired | macro-unconditional | branches excluding | calls and indirects] | ... 333 more new events ... | | % perf stat -e br_inst_exec.all_direct_jmp true
Can you specify the qualifiers like ':k' or ':ku' with the events on Intel ? to only monitor kernel or user ? Or do they need some additional JSON entries ?
With the above events file, I get "invalid event" for 'PM_INST_CMPL:u'
| | Performance counter stats for 'true': | | 6,817 cpu/br_inst_exec.all_direct_jmp/ | | 0.003503212 seconds time elapsed | | One nice feature is that a pointer to the specification update is now | included in the description, which will hopefully clear up many problems: | | % perf list | ... | mem_load_uops_l3_hit_retired.xsnp_hit [Retired load uops which | data sources were L3 | and cross-core snoop | hits in on-pkg core | cache. Supports address | when precise. Spec | update: HSM26, HSM30 | (Precise event)] | ... | | | -Andi | -- | To unsubscribe from this list: send the line "unsubscribe linux-kernel" in | the body of a message to [email protected] | More majordomo info at http://vger.kernel.org/majordomo-info.html | Please read the FAQ at http://www.tux.org/lkml/
Thanks for testing.
Quoted 6 lines · Sukadev Bhattiprolu
| The JSON format and perf parser has some minor Intelisms, but they | are simple and small and optional. It's easy to extend, so it would be | possible to use it for other CPUs too, add different pmu attributes, and | add new download sites to the downloader tool.Is there a minimal set of JSON entries an architecture would need ?
That should be enough, assuming the EventCode is enough to select the event.
Quoted 19 lines
I tried the following on Power
[ { "EventCode": "2", "EventName": "PM_INST_CMPL", "BriefDescription": "Instructions completed", "PublicDescription": "Number of PPC instructions finished", }, { "EventCode": "0x1E", "EventName": "PM_CYC", "BriefDescription": "Cycles completed", "PublicDescription": "Number of PPC cycles finished", } ] /tmp/perf record --events-file=/tmp/power8.json -e PM_INST_CMPL sleep 1
Quoted 3 lines
works, but for some TBD reason,
/tmp/perflist --events-file=/tmp/power8.json doesn't listPM_INST_CMPL.
Works for me with your input file:
% perf list --events-file t.json
...
pm_cyc [Cycles completed]
pm_inst_cmpl [Instructions completed]
Quoted 2 lines
Another observation was that the order of --events-file and -e is significant. Maybe worth a note in the man page.
Will add.
Quoted 4 lines
Can you specify the qualifiers like ':k' or ':ku' with the events on Intel ?
to only monitor kernel or user ? Or do they need some additional JSON entries ?
They can be all specified.
Quoted 1 line
With the above events file, I get "invalid event" for 'PM_INST_CMPL:u'
It works in new style syntax, like
perf stat --events-file t.json -e cpu/pm_inst_cmpl/u ls
Thanks,
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Andi Kleen [[email protected]] wrote:
| Works for me with your input file: | | % perf list --events-file t.json | ... | pm_cyc [Cycles completed] | pm_inst_cmpl [Instructions completed]
Ah, lower case.
| | > With the above events file, I get "invalid event" for 'PM_INST_CMPL:u' | | It works in new style syntax, like | | perf stat --events-file t.json -e cpu/pm_inst_cmpl/u ls
Yes, thanks.
Sukadev
Quoted 12 lines · Andi Kleen
Should be ready for merge now. Please consider.
[v2: Review feedback addressed and some minor improvements] [v3: More review feedback addressed and handle test failures better. Ported to latest tip/core.]
[v4: Addressed Namhyung's feedback]
[v5: Rebase to latest tree. Minor description update.]
[v6: Rebase. Add acked by from Namhyung and address feedback. Some minor fixes. Should be good to go now I hope. The period patch was dropped, as that is already handled. I added an extra patch for a --quiet argument for perf list]
[v7: Just rebase to latest tip/core. Should be ready to merge.]
please remove 'vX' suffixes from patch subjects
jirka