Skip to contentSkip to conversation

perf: Add support for full Intel event lists

PATCH In review Andi Kleen 3 participants 27 messages Mar 5, 2014 – Mar 14, 2014 perf events

lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

perf has high level events which are useful in many cases. However there are some tuning situations where low level events in the CPU are needed. Traditionally this required specifying the event in raw form (very awkward) or using non standard frontends like ocperf or patching in libpfm.

Intel CPUs can have very large event files (Haswell has ~336 core events, much more if you add uncore or all the offcore combinations), which is too large to describe through the kernel interface. It would require tying up significant amounts of unswappable memory for this.

oprofile always had separate event list files that were maintained by the CPU vendors. The oprofile events were shipped with the tool. The Intel events get updated regularly, for example to add references to the specification updates or add new events.

Unfortunately oprofile usually did not keep up with these updates, so the events in oprofile were often out of date. In addition it ties up quite a bit of disk space, mostly for CPUs you don't have.

This patch kit implements another mechanism that avoids these problems. Intel releases the event lists for CPUs in a standardized JSON format on a download server.

I implemented an automatic downloader to get the event file for the current CPU. The events are stored in ~/.events.
Then perf adds a parser that converts the JSON format into perf event aliases, which then can be used directly as any other perf event.

The parsing is done using a simple existing JSON library.

The events are still abstracted for perf, but the abstraction mechanism is through the downloaded file instead of through the kernel.

The JSON format and perf parser has some minor Intelisms, but they are simple and small and optional. It's easy to extend, so it would be possible to use it for other CPUs too, add different pmu attributes, and add new download sites to the downloader tool.

Currently only core events are supported, uncore may come at a later point. No kernel changes, all code in perf user tools only.

Some of the parser files are partially shared with separate event parser library and are thus 2-clause BSD licensed.

Patches also available from
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc perf/json

Example output:

% perf download
Downloading models file
Downloading readme.txt

2014-03-05 10:39:33 URL:https://download.01.org/perfmon/readme.txt [10320/10320] -> "readme.txt" [1]
2014-03-05 10:39:34 URL:https://download.01.org/perfmon/mapfile.csv [1207/1207] -> "mapfile.csv" [1]

Downloading events file
% perf list
...
br_inst_exec.all_branches [Speculative and retired
branches]
br_inst_exec.all_conditional [Speculative and retired
macro-conditional branches]
br_inst_exec.all_direct_jmp [Speculative and retired
macro-unconditional branches excluding calls and indirects]
... 333 more new events ...

% perf stat -e br_inst_exec.all_direct_jmp true

Performance counter stats for 'true':

             6,817      cpu/br_inst_exec.all_direct_jmp/                                   

       0.003503212 seconds time elapsed

One nice feature is that a pointer to the specification update is now included in the description, which will hopefully clear up many problems:

% perf list
...
mem_load_uops_l3_hit_retired.xsnp_hit [Retired load uops which
data sources were L3 and cross-core snoop hits in on-pkg core cache. Supports address when precise. Spec
update: HSM26, HSM30 (Precise event)]
...

-Andi

author
#1 lore raw

PATCH 1/8 perf, tools: Add jsmn `jasmine' JSON parser

From: Andi Kleen <[email protected]>

I need a JSON parser. This adds the simplest JSON parser I could find -- Serge Zaitsev's jsmn `jasmine' -- to the perf library. I merely converted it to (mostly) Linux style and added support for non 0 terminated input.

The parser is quite straight forward and does not copy any data, just returns tokens with offsets into the input buffer. So it's relatively efficient and simple to use.

The code is not fully checkpatch clean, but I didn't want to completely fork the upstream code.

Original source: http://zserge.bitbucket.org/jsmn.html

In addition I added a simple wrapper that mmaps a json file and provides some straight forward access functions.

Used in follow-on patches to parse event files.

---
 tools/perf/Makefile.perf |   4 +
 tools/perf/util/jsmn.c   | 297 +++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/jsmn.h   |  65 +++++++++++
 tools/perf/util/json.c   | 156 +++++++++++++++++++++++++
 tools/perf/util/json.h   |   8 ++
 5 files changed, 530 insertions(+)
 create mode 100644 tools/perf/util/jsmn.c
 create mode 100644 tools/perf/util/jsmn.h
 create mode 100644 tools/perf/util/json.c
 create mode 100644 tools/perf/util/json.h
tools/perf/Makefile.perf +4 -0
index 1f7ec48..70e6e5a 100644
@@ -301,6 +301,8 @@ LIB_H += ui/progress.h
 LIB_H += ui/util.h
 LIB_H += ui/ui.h
 LIB_H += util/data.h
+LIB_H += util/jsmn.h
+LIB_H += util/json.h
 
 LIB_OBJS += $(OUTPUT)util/abspath.o
 LIB_OBJS += $(OUTPUT)util/alias.o
@@ -374,6 +376,8 @@ LIB_OBJS += $(OUTPUT)util/stat.o
 LIB_OBJS += $(OUTPUT)util/record.o
 LIB_OBJS += $(OUTPUT)util/srcline.o
 LIB_OBJS += $(OUTPUT)util/data.o
+LIB_OBJS += $(OUTPUT)util/jsmn.o
+LIB_OBJS += $(OUTPUT)util/json.o
 
 LIB_OBJS += $(OUTPUT)ui/setup.o
 LIB_OBJS += $(OUTPUT)ui/helpline.o
tools/perf/util/jsmn.c new file +297 -0

297 lines not on this page · raw message

tools/perf/util/jsmn.h new file +65 -0

65 lines not on this page · raw message

tools/perf/util/json.c new file +156 -0

156 lines not on this page · raw message

tools/perf/util/json.h new file +8 -0
new file mode 100644
index 0000000..0a34aa2
@@ -0,0 +1,8 @@
+#include "jsmn.h"
+
+jsmntok_t *parse_json(const char *fn, char **map, size_t *size, int *len);
+void free_json(char *map, size_t size, jsmntok_t *tokens);
+int json_line(char *map, jsmntok_t *t);
+const char *json_name(jsmntok_t *t);
+int json_streq(char *map, jsmntok_t *t, const char *s);
+int json_len(jsmntok_t *t);
Signature · 1 line
1.8.5.3
#2 lore raw

Re: [PATCH 1/8] perf, tools: Add jsmn `jasmine' JSON parser

Quoted 2 lines · Andi Kleen

From: Andi Kleen <[email protected]>

SNIP

tools/perf/util/json.h

Quoted 16 lines
tools/perf/util/json.c new file +10 -0
+{
+	return t->end - t->start;
+}
+
+/* Is string t equal to s? */
+int json_streq(char *map, jsmntok_t *t, const char *s)
+{
+	unsigned len = t->end - t->start;
+	return len == strlen(s) && !strncasecmp(map + t->start, s, len);
+}
tools/perf/util/json.h new file +0 -0
new file mode 100644
index 0000000..0a34aa2
@@ -0,0 +1,8 @@

missing

#ifndef JSON_H
#define JSON_H

Quoted 11 lines
tools/perf/util/json.h new file +8 -0
+#include "jsmn.h"
+
+jsmntok_t *parse_json(const char *fn, char **map, size_t *size, int *len);
+void free_json(char *map, size_t size, jsmntok_t *tokens);
+int json_line(char *map, jsmntok_t *t);
+const char *json_name(jsmntok_t *t);
+int json_streq(char *map, jsmntok_t *t, const char *s);
+int json_len(jsmntok_t *t);
Signature · 1 line
1.8.5.3
#3 lore raw

Re: [PATCH 1/8] perf, tools: Add jsmn `jasmine' JSON parser

In reply to Andi Kleen
Quoted 2 lines · Andi Kleen

From: Andi Kleen <[email protected]>

SNIP

Quoted 38 lines
tools/perf/util/jsmn.h new file +1 -0
+#endif /* __JSMN_H_ */
tools/perf/util/json.c new file +31 -0
new file mode 100644
index 0000000..48201114
@@ -0,0 +1,156 @@
+/* Parse JSON files using the JSMN parser. */
+
+/*
+ * Copyright (c) 2014, Intel Corporation
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+ * OF THE POSSIBILITY OF SUCH DAMAGE.
+*/
+
+#include <stdlib.h>

tools/perf/util/json.c line 41

Quoted 10 lines
tools/perf/util/json.c new file +10 -0
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/fcntl.h>
+#include <stdio.h>
+#include <unistd.h>
+#include "jsmn.h"
+#include "json.h"
+
+#define roundup(x, y) (((x) + (y) - 1) & ~((y) - 1))

this one is defined in util/include/linux/kernel.h

jirka

#4 lore raw

Re: [PATCH 1/8] perf, tools: Add jsmn `jasmine' JSON parser

In reply to Andi Kleen
Quoted 2 lines · Andi Kleen

From: Andi Kleen <[email protected]>

SNIP

Quoted 10 lines
tools/perf/util/json.c new file +10 -0
+jsmntok_t *parse_json(const char *fn, char **map, size_t *size, int *len)
+{
+	jsmn_parser parser;
+	jsmntok_t *tokens;
+	jsmnerr_t res;
+	unsigned sz;
+
+	*map = mapfile(fn, size);
+	if (!*map)
+		return NULL;

tools/perf/util/json.c line 95

Quoted 10 lines
tools/perf/util/json.c new file +10 -0
+	/* Heuristic */
+	sz = *size * 16;
+	tokens = malloc(sz);
+	if (!tokens)
+		goto error;
+	jsmn_init(&parser);
+	res = jsmn_parse(&parser, *map, *size, tokens,
+			 sz / sizeof(jsmntok_t));
+	if (res != JSMN_SUCCESS) {
+		fprintf(stderr, "%s: json error %d\n", fn, res);

use pr_err

jirka

author
#5 lore raw

Re: [PATCH 1/8] perf, tools: Add jsmn `jasmine' JSON parser

Quoted 20 lines · Jiri Olsa
Quoted 2 lines · Andi Kleen

From: Andi Kleen <[email protected]>

SNIP

Quoted 14 lines
tools/perf/util/json.c new file +14 -0
+jsmntok_t *parse_json(const char *fn, char **map, size_t *size, int *len)
+{
+	jsmn_parser parser;
+	jsmntok_t *tokens;
+	jsmnerr_t res;
+	unsigned sz;
+
+	*map = mapfile(fn, size);
+	if (!*map)
+		return NULL;
+	/* Heuristic */
+	sz = *size * 16;
+	tokens = malloc(sz);
+	if (!tokens)
Quoted 8 lines

tools/perf/util/json.c line 95

Quoted 6 lines
tools/perf/util/json.c new file +6 -0
+		goto error;
+	jsmn_init(&parser);
+	res = jsmn_parse(&parser, *map, *size, tokens,
+			 sz / sizeof(jsmntok_t));
+	if (res != JSMN_SUCCESS) {
+		fprintf(stderr, "%s: json error %d\n", fn, res);

use pr_err

dito.
-Andi

#6 lore raw

Re: [PATCH 1/8] perf, tools: Add jsmn `jasmine' JSON parser

In reply to Andi Kleen
Quoted 2 lines · Andi Kleen

From: Andi Kleen <[email protected]>

SNIP

Quoted 42 lines
tools/perf/util/jsmn.h new file +1 -0
+#endif /* __JSMN_H_ */
tools/perf/util/json.c new file +35 -0
new file mode 100644
index 0000000..48201114
@@ -0,0 +1,156 @@
+/* Parse JSON files using the JSMN parser. */
+
+/*
+ * Copyright (c) 2014, Intel Corporation
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+ * OF THE POSSIBILITY OF SUCH DAMAGE.
+*/
+
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/fcntl.h>

tools/perf/util/json.c line 45

Quoted 10 lines
tools/perf/util/json.c new file +10 -0
+#include <stdio.h>
+#include <unistd.h>
+#include "jsmn.h"
+#include "json.h"
+
+#define roundup(x, y) (((x) + (y) - 1) & ~((y) - 1))
+
+static char *mapfile(const char *fn, size_t *size)
+{
+	unsigned ps = sysconf(_SC_PAGESIZE);

you can use page_size instead

Quoted 13 lines
tools/perf/util/json.c new file +13 -0
+	struct stat st;
+	char *map = NULL;
+	int err;
+	int fd = open(fn, O_RDONLY);
+
+	if (fd < 0)
+		return NULL;
+	err = fstat(fd, &st);
+	if (err < 0)
+		goto out;
+	*size = st.st_size;
+	map = mmap(NULL, roundup(st.st_size, ps),
+		   PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);

tools/perf/util/json.c line 68

Quoted 10 lines
tools/perf/util/json.c new file +10 -0
+	if (map == (char *)MAP_FAILED)
+		map = NULL;
+out:
+	close(fd);
+	return map;
+}
+
+static void unmapfile(char *map, size_t size)
+{
+	unsigned ps = sysconf(_SC_PAGESIZE);

ditto

jirka

author
#7 lore raw

Re: [PATCH 1/8] perf, tools: Add jsmn `jasmine' JSON parser

Quoted 5 lines

tools/perf/util/json.c line 45

Quoted 3 lines
tools/perf/util/json.c new file +3 -0
+static char *mapfile(const char *fn, size_t *size)
+{
+	unsigned ps = sysconf(_SC_PAGESIZE);

you can use page_size instead

I decided not to do this as these are stand alone files.

-Andi

#8 lore raw

Re: [PATCH 1/8] perf, tools: Add jsmn `jasmine' JSON parser

In reply to Andi Kleen
Quoted 1 line · Andi Kleen

From: Andi Kleen <[email protected]>

SNIP

Quoted 4 lines
tools/perf/util/json.c new file +4 -0
+/* Return line number of a jsmn token */
+int json_line(char *map, jsmntok_t *t)
+{
+	return countchar(map, '\n', t->start) + 1;

tools/perf/util/json.c line 137

Quoted 10 lines
tools/perf/util/json.c new file +10 -0
+}
+
+static const char *jsmn_types[] = {
+	[JSMN_PRIMITIVE] = "primitive",
+	[JSMN_ARRAY] = "array",
+	[JSMN_OBJECT] = "object",
+	[JSMN_STRING] = "string"
+};
+
+#define NELE(x) (sizeof(x) / sizeof(*(x)))

use ARRAY_SIZE

jirka

author
#9 lore raw

PATCH 2/8 perf, tools: Add support for reading JSON event files

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Add a parser for Intel style JSON event files. This allows to use an Intel event list directly with perf. The Intel event lists can be quite large and are too big to store in unswappable kernel memory.

The parser code knows how to convert the JSON fields
to perf fields. The conversion code is straight forward. It knows (very little) Intel specific information, and can be easily extended to handle fields for other CPUs.

The parser code is partially shared with an independent parsing library, which is 2-clause BSD licenced. To avoid any conflicts I marked those files as BSD licenced too. As part of perf they become GPLv2.

The events are handled using the existing alias machinery.

We output the BriefDescription in perf list.

Right now the json file can be specified as an argument to perf stat/record/list. Followon patches will automate this.

---
 tools/perf/Documentation/perf-list.txt   |   6 +
 tools/perf/Documentation/perf-record.txt |   3 +
 tools/perf/Documentation/perf-stat.txt   |   3 +
 tools/perf/Makefile.perf                 |   2 +
 tools/perf/builtin-list.c                |   2 +
 tools/perf/builtin-record.c              |   3 +
 tools/perf/builtin-stat.c                |   2 +
 tools/perf/util/jevents.c                | 248 +++++++++++++++++++++++++++++++
 tools/perf/util/jevents.h                |   3 +
 tools/perf/util/pmu.c                    | 141 ++++++++++++++----
 tools/perf/util/pmu.h                    |   2 +
 11 files changed, 386 insertions(+), 29 deletions(-)
 create mode 100644 tools/perf/util/jevents.c
 create mode 100644 tools/perf/util/jevents.h
tools/perf/Documentation/perf-list.txt +6 -0
index 6fce6a6..d399e04 100644
@@ -15,6 +15,12 @@ DESCRIPTION
 This command displays the symbolic event types which can be selected in the
 various perf commands with the -e option.
 
+OPTIONS
+-------
+--json-file=::
+Specify JSON event list file to use for parsing events.
+
+
 [[EVENT_MODIFIERS]]
 EVENT MODIFIERS
 ---------------
tools/perf/Documentation/perf-record.txt +3 -0
index c71b0f3..8a1e0d9 100644
@@ -213,6 +213,9 @@ if combined with -a or -C options.
 After starting the program, wait msecs before measuring. This is useful to
 filter out the startup phase of the program, which is often very different.
 
+--json-file=::
+Specify JSON event list file to use for parsing events.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/Documentation/perf-stat.txt +3 -0
index 29ee857..a946283 100644
@@ -142,6 +142,9 @@ filter out the startup phase of the program, which is often very different.
 
 Print statistics of transactional execution if supported.
 
+--json-file=::
+Specify JSON event list file to use for parsing events.
+
 EXAMPLES
 --------
 
tools/perf/Makefile.perf +2 -0
index 70e6e5a..46f1894 100644
@@ -303,6 +303,7 @@ LIB_H += ui/ui.h
 LIB_H += util/data.h
 LIB_H += util/jsmn.h
 LIB_H += util/json.h
+LIB_H += util/jevents.h
 
 LIB_OBJS += $(OUTPUT)util/abspath.o
 LIB_OBJS += $(OUTPUT)util/alias.o
@@ -378,6 +379,7 @@ LIB_OBJS += $(OUTPUT)util/srcline.o
 LIB_OBJS += $(OUTPUT)util/data.o
 LIB_OBJS += $(OUTPUT)util/jsmn.o
 LIB_OBJS += $(OUTPUT)util/json.o
+LIB_OBJS += $(OUTPUT)util/jevents.o
 
 LIB_OBJS += $(OUTPUT)ui/setup.o
 LIB_OBJS += $(OUTPUT)ui/helpline.o
tools/perf/builtin-list.c +2 -0
index 011195e..ca3dd18 100644
@@ -20,6 +20,8 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 {
 	int i;
 	const struct option list_options[] = {
+		OPT_STRING(0, "json-file", &json_file, "json file",
+			   "Read event json file"),
 		OPT_END()
 	};
 	const char * const list_usage[] = {
tools/perf/builtin-record.c +3 -0
index eb524f9..b0cf944 100644
@@ -25,6 +25,7 @@
 #include "util/cpumap.h"
 #include "util/thread_map.h"
 #include "util/data.h"
+#include "util/pmu.h"
 
 #include <unistd.h>
 #include <sched.h>
@@ -910,6 +911,8 @@ const struct option record_options[] = {
 		    "sample transaction flags (special events only)"),
 	OPT_BOOLEAN(0, "per-thread", &record.opts.target.per_thread,
 		    "use per-thread mmaps"),
+	OPT_STRING(0, "json-file", &json_file, "json file",
+		   "Read event json file"),
 	OPT_END()
 };
 
tools/perf/builtin-stat.c +2 -0
index 8b0e1c9..e521ba0 100644
@@ -1661,6 +1661,8 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
 		     "aggregate counts per physical processor core", AGGR_CORE),
 	OPT_UINTEGER('D', "delay", &initial_delay,
 		     "ms to wait before starting measurement after program start"),
+	OPT_STRING(0, "json-file", &json_file, "json file",
+		   "Read event json file"),
 	OPT_END()
 	};
 	const char * const stat_usage[] = {
tools/perf/util/jevents.c new file +248 -0

248 lines not on this page · raw message

tools/perf/util/jevents.h new file +3 -0
new file mode 100644
index 0000000..4c2b879
@@ -0,0 +1,3 @@
+int json_events(const char *fn,
+		int (*func)(void *data, char *name, char *event, char *desc),
+		void *data);
tools/perf/util/pmu.c +112 -29

230 lines not on this page · raw message

tools/perf/util/pmu.h +2 -0
index 8b64125..62255bb 100644
@@ -46,4 +46,6 @@ void print_pmu_events(const char *event_glob, bool name_only);
 bool pmu_have_event(const char *pname, const char *name);
 
 int perf_pmu__test(void);
+
+extern const char *json_file;
 #endif /* __PMU_H */
Signature · 1 line
1.8.5.3
#10 lore raw

Re: [PATCH 2/8] perf, tools: Add support for reading JSON event files

Quoted 24 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Add a parser for Intel style JSON event files. This allows to use an Intel event list directly with perf. The Intel event lists can be quite large and are too big to store in unswappable kernel memory.

The parser code knows how to convert the JSON fields
to perf fields. The conversion code is straight forward. It knows (very little) Intel specific information, and can be easily extended to handle fields for other CPUs.

The parser code is partially shared with an independent parsing library, which is 2-clause BSD licenced. To avoid any conflicts I marked those files as BSD licenced too. As part of perf they become GPLv2.

The events are handled using the existing alias machinery.

We output the BriefDescription in perf list.

Right now the json file can be specified as an argument to perf stat/record/list. Followon patches will automate this.

Quoted 12 lines
---
 tools/perf/Documentation/perf-list.txt   |   6 +
 tools/perf/Documentation/perf-record.txt |   3 +
 tools/perf/Documentation/perf-stat.txt   |   3 +
 tools/perf/Makefile.perf                 |   2 +
 tools/perf/builtin-list.c                |   2 +
 tools/perf/builtin-record.c              |   3 +
 tools/perf/builtin-stat.c                |   2 +
 tools/perf/util/jevents.c                | 248 +++++++++++++++++++++++++++++++
 tools/perf/util/jevents.h                |   3 +
 tools/perf/util/pmu.c                    | 141 ++++++++++++++----
 tools/perf/util/pmu.h                    |   2 +

please put the alias code factoring (perf_pmu__new_alias) and print_pmu_events changes to separate commits

jirka

author
#11 lore raw

PATCH 3/8 perf, tools: Automatically look for event file name for cpu

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

When no JSON event file is specified automatically look for a suitable file in ~/.events. A "perf download" can automatically add files there for the current CPUs.

This does not include the actual event files with perf, but they can be automatically downloaded instead (implemented in the next patch)

This has the advantage that the events can be always uptodate, because they are freshly downloaded. In oprofile we always had problems with out of date or incomplete events files.

The event file format is per architecture, but can be extended for other architectures.

---
 tools/perf/arch/x86/Makefile      |  1 +
 tools/perf/arch/x86/util/cpustr.c | 35 +++++++++++++++++++++++++++++++++++
 tools/perf/util/jevents.c         | 22 ++++++++++++++++++++++
 tools/perf/util/jevents.h         |  1 +
 tools/perf/util/pmu.c             |  2 +-
 5 files changed, 60 insertions(+), 1 deletion(-)
 create mode 100644 tools/perf/arch/x86/util/cpustr.c
tools/perf/arch/x86/Makefile +1 -0
index 1641542..0efeb14 100644
@@ -14,4 +14,5 @@ LIB_OBJS += $(OUTPUT)arch/$(ARCH)/tests/dwarf-unwind.o
 endif
 LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/header.o
 LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/tsc.o
+LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/cpustr.o
 LIB_H += arch/$(ARCH)/util/tsc.h
tools/perf/arch/x86/util/cpustr.c new file +35 -0
new file mode 100644
index 0000000..0fb987a
@@ -0,0 +1,35 @@
+#define _GNU_SOURCE 1
+#include <stdio.h>
+#include <stdlib.h>
+#include "../../util/jevents.h"
+
+char *get_cpu_str(void)
+{
+	char *line = NULL;
+	size_t llen = 0;
+	int found = 0, n;
+	char vendor[30];
+	int model, fam;
+	char *res = NULL;
+	FILE *f = fopen("/proc/cpuinfo", "r");
+
+	if (!f)
+		return NULL;
+	while (getline(&line, &llen, f) > 0) {
+		if (sscanf(line, "vendor_id : %30s", vendor) == 1)
+			found++;
+		else if (sscanf(line, "model : %d", &model) == 1)
+			found++;
+		else if (sscanf(line, "cpu family : %d", &fam) == 1)
+			found++;
+		if (found == 3) {
+			n = asprintf(&res, "%s-%d-%X-core", vendor, fam, model);
+			if (n < 0)
+				res = NULL;
+			break;
+		}
+	}
+	free(line);
+	fclose(f);
+	return res;
+}
tools/perf/util/jevents.c +22 -0
index 0173b68..a3c776a 100644
@@ -33,10 +33,30 @@
 #include <errno.h>
 #include <string.h>
 #include <ctype.h>
+#include "cache.h"
 #include "jsmn.h"
 #include "json.h"
 #include "jevents.h"
 
+__attribute__((weak)) char *get_cpu_str(void)
+{
+	return NULL;
+}
+
+static const char *json_default_name(void)
+{
+	char *home = getenv("HOME");
+	char *idstr = get_cpu_str();
+	char *res = NULL;
+
+	if (home && idstr)
+		res = mkpath("%s/.events/%s.json",
+			     home,
+			     idstr);
+	free(idstr);
+	return res;
+}
+
 static void addfield(char *map, char **dst, const char *sep,
 		     const char *a, jsmntok_t *bt)
 {
@@ -171,6 +191,8 @@ int json_events(const char *fn,
 	int i, j, len;
 	char *map;
 
+	if (!fn)
+		fn = json_default_name();
 	tokens = parse_json(fn, &map, &size, &len);
 	if (!tokens)
 		return -EIO;
tools/perf/util/jevents.h +1 -0
index 4c2b879..6a377a8 100644
@@ -1,3 +1,4 @@
 int json_events(const char *fn,
 		int (*func)(void *data, char *name, char *event, char *desc),
 		void *data);
+char *get_cpu_str(void);
tools/perf/util/pmu.c +1 -1
index 831f525..5d69c29 100644
@@ -433,7 +433,7 @@ static struct perf_pmu *pmu_lookup(const char *name)
 	if (pmu_aliases(name, &aliases))
 		return NULL;
 
-	if (!strcmp(name, "cpu") && json_file)
+	if (!strcmp(name, "cpu"))
 		json_events(json_file, add_alias, &aliases);
 
 	if (pmu_type(name, &type))
Signature · 1 line
1.8.5.3
#12 lore raw

Re: [PATCH 3/8] perf, tools: Automatically look for event file name for cpu

Quoted 2 lines · Andi Kleen

From: Andi Kleen <[email protected]>

SNIP

Quoted 17 lines
tools/perf/arch/x86/Makefile +1 -0
index 1641542..0efeb14 100644
@@ -14,4 +14,5 @@ LIB_OBJS += $(OUTPUT)arch/$(ARCH)/tests/dwarf-unwind.o
 endif
 LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/header.o
 LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/tsc.o
+LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/cpustr.o
 LIB_H += arch/$(ARCH)/util/tsc.h
tools/perf/arch/x86/util/cpustr.c new file +0 -0
new file mode 100644
index 0000000..0fb987a

tools/perf/arch/x86/util/cpustr.c line 1

Quoted 1 line
tools/perf/arch/x86/util/cpustr.c new file +1 -0
@@ -0,0 +1,35 @@
+#define _GNU_SOURCE 1

_GNU_SOURCE is defined by default

jirka

author
#13 lore raw

PATCH 4/8 perf, tools: Add perf download to download event files

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Add a downloader to automatically download the right files from a download site.

This is implemented as a script calling wget, similar to perf archive. The perf driver automatically calls the right binary. The downloader is extensible, but currently only implements an Intel event download. It would be straightforward to add other sites too for other vendors.

The downloaded event files are put into ~/.events, where the builtin event parser in util/* can find them automatically.

---
 tools/perf/Documentation/perf-download.txt | 25 +++++++++++++++
 tools/perf/Documentation/perf-list.txt     | 12 ++++++-
 tools/perf/Makefile.perf                   |  5 ++-
 tools/perf/perf-download.sh                | 50 ++++++++++++++++++++++++++++++
 4 files changed, 90 insertions(+), 2 deletions(-)
 create mode 100644 tools/perf/Documentation/perf-download.txt
 create mode 100755 tools/perf/perf-download.sh
tools/perf/Documentation/perf-download.txt new file +25 -0
new file mode 100644
index 0000000..73570e2
@@ -0,0 +1,25 @@
+perf-download(1)
+===============
+
+NAME
+----
+perf-download - Download event files for current CPU.
+
+SYNOPSIS
+--------
+[verse]
+'perf download' [vendor-family-model]
+
+DESCRIPTION
+-----------
+This command automatically downloads event list for the current CPU and
+stores them in $HOME/.event. The other tools automatically look
+for them there. The CPU can be also specified at the command line.
+The downloading is done using http through wget.
+
+The user should regularly call this to download updated event lists
+for the CPU.
+
+Note the downloaded files are stored per user, so if you use
+perf as normal user or with sudo you have to run perf download
+for both or specify --json-file for the users manually.
tools/perf/Documentation/perf-list.txt +11 -1
index d399e04..e8d5153 100644
@@ -61,6 +61,16 @@ Sampling). Examples to use IBS:
  perf record -a -e r076:p ...          # same as -e cpu-cycles:p
  perf record -a -e r0C1:p ...          # use ibs op counting micro-ops
 
+PER CPU EVENT LISTS
+-------------------
+
+For some CPUs (particularly modern Intel CPUs) "perf download" can
+download additional CPU specific event definitions, which then
+become visible in perf list and available in the other perf tools.
+
+This obsoletes the raw event description method described below
+for most cases.
+
 RAW HARDWARE EVENT DESCRIPTOR
 -----------------------------
 Even when an event is not available in a symbolic form within perf right now,
@@ -123,6 +133,6 @@ types specified.
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-top[1],
-linkperf:perf-record[1],
+linkperf:perf-record[1], linkperf:perf-download[1],
 http://www.intel.com/Assets/PDF/manual/253669.pdf[Intel® 64 and IA-32 Architectures Software Developer's Manual Volume 3B: System Programming Guide],
 http://support.amd.com/us/Processor_TechDocs/24593_APM_v2.pdf[AMD64 Architecture Programmer’s Manual Volume 2: System Programming]
tools/perf/Makefile.perf +4 -1
index 46f1894..8c4cc73 100644
@@ -126,6 +126,7 @@ PYRF_OBJS =
 SCRIPT_SH =
 
 SCRIPT_SH += perf-archive.sh
+SCRIPT_SH += perf-download.sh
 
 grep-libs = $(filter -l%,$(1))
 strip-libs = $(filter-out -l%,$(1))
@@ -866,6 +867,8 @@ install-bin: all install-gtk
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)'
 	$(call QUIET_INSTALL, perf-archive) \
 		$(INSTALL) $(OUTPUT)perf-archive -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)'
+	$(call QUIET_INSTALL, perf-download) \
+		$(INSTALL) $(OUTPUT)perf-download -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)'
 ifndef NO_LIBPERL
 	$(call QUIET_INSTALL, perl-scripts) \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl/Perf-Trace-Util/lib/Perf/Trace'; \
@@ -911,7 +914,7 @@ config-clean:
 	@$(MAKE) -C config/feature-checks clean >/dev/null
 
 clean: $(LIBTRACEEVENT)-clean $(LIBAPIKFS)-clean config-clean
-	$(call QUIET_CLEAN, core-objs)  $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf.o $(LANG_BINDINGS) $(GTK_OBJS)
+	$(call QUIET_CLEAN, core-objs)  $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)/perf-download $(OUTPUT)perf.o $(LANG_BINDINGS) $(GTK_OBJS)
 	$(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf
 	$(call QUIET_CLEAN, core-gen)   $(RM)  *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope* $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)PERF-CFLAGS $(OUTPUT)PERF-FEATURES $(OUTPUT)util/*-bison* $(OUTPUT)util/*-flex*
 	$(QUIET_SUBDIR0)Documentation $(QUIET_SUBDIR1) clean
tools/perf/perf-download.sh new file +50 -0

50 lines not on this page · raw message

Signature · 1 line
1.8.5.3
author
#14 lore raw

PATCH 5/8 perf, tools: Allow events with dot

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

The Intel events use a dot to separate event name and unit mask. Allow dot in names in the scanner, and remove special handling of dot as EOF. Also remove the hack in jevents to replace dot with underscore. This way dotted events can be specified directly by the user.

I'm not fully sure this change to the scanner is correct (what was the dot special case good for?), but I haven't found anything that breaks with it so far at least.

---
 tools/perf/util/jevents.c      | 9 +--------
 tools/perf/util/parse-events.l | 3 +--
 2 files changed, 2 insertions(+), 10 deletions(-)
tools/perf/util/jevents.c +1 -8
index a3c776a..9743d70 100644
@@ -78,15 +78,8 @@ static void addfield(char *map, char **dst, const char *sep,
 
 static void fixname(char *s)
 {
-	for (; *s; s++) {
+	for (; *s; s++)
 		*s = tolower(*s);
-		/*
-		 * Remove '.' for now, until the parser
-		 * can deal with it.
-		 */
-		if (*s == '.')
-			*s = '_';
-	}
 }
 
 static void fixdesc(char *s)
tools/perf/util/parse-events.l +1 -2
index 3432995..709fa3b 100644
@@ -81,7 +81,7 @@ num_dec		[0-9]+
 num_hex		0x[a-fA-F0-9]+
 num_raw_hex	[a-fA-F0-9]+
 name		[a-zA-Z_*?][a-zA-Z0-9_*?]*
-name_minus	[a-zA-Z_*?][a-zA-Z0-9\-_*?]*
+name_minus	[a-zA-Z_*?][a-zA-Z0-9\-_*?.]*
 /* If you add a modifier you need to update check_modifier() */
 modifier_event	[ukhpGHSD]+
 modifier_bp	[rwx]{1,3}
@@ -119,7 +119,6 @@ modifier_bp	[rwx]{1,3}
 			return PE_EVENT_NAME;
 		}
 
-.		|
 <<EOF>>		{
 			BEGIN(INITIAL); yyless(0);
 		}
Signature · 1 line
1.8.5.3
author
#15 lore raw

PATCH 6/8 perf, tools: Query terminal width and use in perf list

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Automatically adapt the now wider and word wrapped perf list output to wider terminals. This requires querying the terminal before the auto pager takes over, and exporting this information from the pager subsystem.

---
 tools/perf/util/cache.h |  1 +
 tools/perf/util/pager.c |  7 ++++++-
 tools/perf/util/pmu.c   | 12 +++++++-----
 3 files changed, 14 insertions(+), 6 deletions(-)
tools/perf/util/cache.h +1 -0
index 7b176dd..d40cd4d 100644
@@ -31,6 +31,7 @@ extern void setup_pager(void);
 extern const char *pager_program;
 extern int pager_in_use(void);
 extern int pager_use_color;
+extern int pager_columns;
 
 char *alias_lookup(const char *alias);
 int split_cmdline(char *cmdline, const char ***argv);
tools/perf/util/pager.c +6 -1
index 3322b84..5a771ab 100644
@@ -1,13 +1,15 @@
 #include "cache.h"
 #include "run-command.h"
 #include "sigchain.h"
+#include <sys/ioctl.h>
 
 /*
  * This is split up from the rest of git so that we can do
  * something different on Windows.
  */
 
-static int spawned_pager;
+int spawned_pager;
+int pager_columns;
 
 static void pager_preexec(void)
 {
@@ -47,9 +49,12 @@ static void wait_for_pager_signal(int signo)
 void setup_pager(void)
 {
 	const char *pager = getenv("PERF_PAGER");
+	struct winsize sz;
 
 	if (!isatty(1))
 		return;
+	if (ioctl(1, TIOCGWINSZ, &sz) == 0)
+		pager_columns = sz.ws_col;
 	if (!pager) {
 		if (!pager_program)
 			perf_config(perf_default_config, NULL);
tools/perf/util/pmu.c +7 -5
index 5d69c29..a26ec85 100644
@@ -9,6 +9,7 @@
 #include "pmu.h"
 #include "parse-events.h"
 #include "cpumap.h"
+#include "cache.h"
 #include "jevents.h"
 
 const char *json_file;
@@ -791,11 +792,12 @@ static void wordwrap(char *s, int start, int max, int corr)
 
 static int get_columns(void)
 {
-	/*
-	 * Should ask the terminal with TIOCGWINSZ here, but we
-	 * need the original fd before the pager.
-	 */
-	return 79;
+	char *s;
+
+	s = getenv("COLUMNS");
+	if (s)
+		return atoi(s);
+	return (pager_columns ? pager_columns : 80) - 2;
 }
 
 void print_pmu_events(const char *event_glob, bool name_only)
Signature · 1 line
1.8.5.3
#16 lore raw

Re: [PATCH 6/8] perf, tools: Query terminal width and use in perf list

Quoted 33 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Automatically adapt the now wider and word wrapped perf list output to wider terminals. This requires querying the terminal before the auto pager takes over, and exporting this information from the pager subsystem.

---
 tools/perf/util/cache.h |  1 +
 tools/perf/util/pager.c |  7 ++++++-
 tools/perf/util/pmu.c   | 12 +++++++-----
 3 files changed, 14 insertions(+), 6 deletions(-)
tools/perf/util/cache.h +1 -0
index 7b176dd..d40cd4d 100644
@@ -31,6 +31,7 @@ extern void setup_pager(void);
 extern const char *pager_program;
 extern int pager_in_use(void);
 extern int pager_use_color;
+extern int pager_columns;
 
 char *alias_lookup(const char *alias);
 int split_cmdline(char *cmdline, const char ***argv);
tools/perf/util/pager.c +0 -0
index 3322b84..5a771ab 100644
@@ -1,13 +1,15 @@
 #include "cache.h"
 #include "run-command.h"

tools/perf/util/pager.c line 11

Quoted 10 lines
tools/perf/util/pager.c +2 -1
 #include "sigchain.h"
+#include <sys/ioctl.h>
 
 /*
  * This is split up from the rest of git so that we can do
  * something different on Windows.
  */
 
-static int spawned_pager;
+int spawned_pager;

this one can stay static

SNIP

Quoted 17 lines

index 5d69c29..a26ec85 100644

tools/perf/util/pmu.c +1 -2
@@ -9,6 +9,7 @@
 #include "pmu.h"
 #include "parse-events.h"
 #include "cpumap.h"
+#include "cache.h"
 #include "jevents.h"
 
 const char *json_file;
@@ -791,11 +792,12 @@ static void wordwrap(char *s, int start, int max, int corr)
 
 static int get_columns(void)
 {
-	/*
-	 * Should ask the terminal with TIOCGWINSZ here, but we

tools/perf/util/pmu.c static void wordwrap(char *s, int start, int max, int corr) · line 801

Quoted 10 lines
tools/perf/util/pmu.c +6 -3
-	 * need the original fd before the pager.
-	 */
-	return 79;
+	char *s;
+
+	s = getenv("COLUMNS");
+	if (s)
+		return atoi(s);
+	return (pager_columns ? pager_columns : 80) - 2;
 }

I think this function should move out of pmu.c
to pager.c perhaps

jirka

author
#17 lore raw

PATCH 7/8 perf, tools, record: Always allow to overide default period

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Fix the logic to allow overriding event default periods with -c or -F on the command line. I'm not sure what the previous if was supposed to do, it didn't work and seemed bogus. I just remove the extra check and unconditionally allow overiding when the user set an option.

This fixed specifying -c / -F with json event list events, which have a default period. It should do the same for trace point events.

---
 tools/perf/util/evsel.c | 18 +++++++-----------
 1 file changed, 7 insertions(+), 11 deletions(-)
tools/perf/util/evsel.c +7 -11
index adc94dd..ca4b976 100644
@@ -561,19 +561,15 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)
 	}
 
 	/*
-	 * We default some events to a 1 default interval. But keep
-	 * it a weak assumption overridable by the user.
+	 * Let the user override any default periods.
 	 */
-	if (!attr->sample_period || (opts->user_freq != UINT_MAX &&
-				     opts->user_interval != ULLONG_MAX)) {
-		if (opts->freq) {
-			perf_evsel__set_sample_bit(evsel, PERIOD);
-			attr->freq		= 1;
-			attr->sample_freq	= opts->freq;
-		} else {
-			attr->sample_period = opts->default_interval;
-		}
+	if (opts->user_freq != UINT_MAX) {
+		perf_evsel__set_sample_bit(evsel, PERIOD);
+		attr->freq		= 1;
+		attr->sample_freq	= opts->freq;
 	}
+	if (opts->user_interval != ULLONG_MAX)
+		attr->sample_period = opts->default_interval;
 
 	/*
 	 * Disable sampling for all group members other
Signature · 1 line
1.8.5.3
#18 lore raw

Re: [PATCH 7/8] perf, tools, record: Always allow to overide default period

Quoted 36 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Fix the logic to allow overriding event default periods with -c or -F on the command line. I'm not sure what the previous if was supposed to do, it didn't work and seemed bogus. I just remove the extra check and unconditionally allow overiding when the user set an option.

This fixed specifying -c / -F with json event list events, which have a default period. It should do the same for trace point events.

---
 tools/perf/util/evsel.c | 18 +++++++-----------
 1 file changed, 7 insertions(+), 11 deletions(-)
tools/perf/util/evsel.c +1 -8
index adc94dd..ca4b976 100644
@@ -561,19 +561,15 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)
 	}
 
 	/*
-	 * We default some events to a 1 default interval. But keep
-	 * it a weak assumption overridable by the user.
+	 * Let the user override any default periods.
 	 */
-	if (!attr->sample_period || (opts->user_freq != UINT_MAX &&
-				     opts->user_interval != ULLONG_MAX)) {
-		if (opts->freq) {
-			perf_evsel__set_sample_bit(evsel, PERIOD);
-			attr->freq		= 1;
-			attr->sample_freq	= opts->freq;

tools/perf/util/evsel.c void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts) · line 572

Quoted 10 lines
tools/perf/util/evsel.c +6 -3
-		} else {
-			attr->sample_period = opts->default_interval;
-		}
+	if (opts->user_freq != UINT_MAX) {
+		perf_evsel__set_sample_bit(evsel, PERIOD);
+		attr->freq		= 1;
+		attr->sample_freq	= opts->freq;
 	}
+	if (opts->user_interval != ULLONG_MAX)
+		attr->sample_period = opts->default_interval;

and this also makes the default frequency not be set ;-)

  $ ./perf record ls
  ...
  $ ./perf report -D | grep SAMPLE
  $

and 'perf test attr' fails for this reason:

  $ ./perf test attr
  13: struct perf_event_attr setup                           :FAILED './tests/attr/test-record-branch-filter-k' - match failure
 FAILED!

jirka

author
#19 lore raw

PATCH 8/8 perf, tools, test: Add test case for alias and JSON parsing

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Add a simple test case to perf test that runs perf download and parses all the events. This needs adding an all event iterator to pmu.c

---
 tools/perf/Makefile.perf        |  1 +
 tools/perf/tests/aliases.c      | 41 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/tests/builtin-test.c |  4 ++++
 tools/perf/tests/tests.h        |  1 +
 tools/perf/util/pmu.c           | 18 ++++++++++++++++++
 tools/perf/util/pmu.h           |  2 ++
 6 files changed, 67 insertions(+)
 create mode 100644 tools/perf/tests/aliases.c
tools/perf/Makefile.perf +1 -0
index 8c4cc73..51bb490 100644
@@ -416,6 +416,7 @@ endif
 LIB_OBJS += $(OUTPUT)tests/code-reading.o
 LIB_OBJS += $(OUTPUT)tests/sample-parsing.o
 LIB_OBJS += $(OUTPUT)tests/parse-no-sample-id-all.o
+LIB_OBJS += $(OUTPUT)tests/aliases.o
 ifndef NO_DWARF_UNWIND
 ifeq ($(ARCH),x86)
 LIB_OBJS += $(OUTPUT)tests/dwarf-unwind.o
tools/perf/tests/aliases.c new file +41 -0

41 lines not on this page · raw message

tools/perf/tests/builtin-test.c +4 -0
index b11bf8a..b77468d 100644
@@ -115,6 +115,10 @@ static struct test {
 		.desc = "Test parsing with no sample_id_all bit set",
 		.func = test__parse_no_sample_id_all,
 	},
+	{
+		.desc = "Test parsing JSON aliases",
+		.func = test__aliases,
+	},
 #if defined(__x86_64__) || defined(__i386__)
 #ifdef HAVE_DWARF_UNWIND_SUPPORT
 	{
tools/perf/tests/tests.h +1 -0
index a24795c..d090866 100644
@@ -41,6 +41,7 @@ int test__sample_parsing(void);
 int test__keep_tracking(void);
 int test__parse_no_sample_id_all(void);
 int test__dwarf_unwind(void);
+int test__aliases(void);
 
 #if defined(__x86_64__) || defined(__i386__)
 #ifdef HAVE_DWARF_UNWIND_SUPPORT
tools/perf/util/pmu.c +18 -0
index a26ec85..652b025 100644
@@ -879,3 +879,21 @@ bool pmu_have_event(const char *pname, const char *name)
 	}
 	return false;
 }
+
+int pmu_iterate_events(int (*func)(const char *name))
+{
+	int ret = 0;
+	struct perf_pmu *pmu;
+	struct perf_pmu_alias *alias;
+
+	perf_pmu__find("cpu"); /* Load PMUs */
+	pmu = NULL;
+	while ((pmu = perf_pmu__scan(pmu)) != NULL) {
+		list_for_each_entry(alias, &pmu->aliases, list) {
+			ret = func(alias->name);
+			if (ret != 0)
+				break;
+		}
+	}
+	return ret;
+}
tools/perf/util/pmu.h +2 -0
index 62255bb..b885c55 100644
@@ -47,5 +47,7 @@ bool pmu_have_event(const char *pname, const char *name);
 
 int perf_pmu__test(void);
 
+int pmu_iterate_events(int (*func)(const char *name));
+
 extern const char *json_file;
 #endif /* __PMU_H */
Signature · 1 line
1.8.5.3
#20 lore raw

Re: [PATCH 8/8] perf, tools, test: Add test case for alias and JSON parsing

Quoted 4 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Add a simple test case to perf test that runs perf download and parses all the events. This needs adding an all event iterator to pmu.c

[jolsa@krava perf]$ ./perf test -vvv 25
25: Test parsing JSON aliases :
--- start ---
invalid or unsupported event: 'energy-gpu'
Parsed 522 events: ---- end ----
Test parsing JSON aliases: FAILED!

I think the problem is in parse_events_fixup, which assumes the 'cpu' pmu globaly for all events

jirka

author
#21 lore raw

Re: [PATCH 8/8] perf, tools, test: Add test case for alias and JSON parsing

Quoted 2 lines

I think the problem is in parse_events_fixup, which assumes the 'cpu' pmu globaly for all events

I investigated this now. The problem is that the event is listed, but cannot be set up. I saw it also with 'data_writes' on another system.

That's a generic problem, the test just exposes it. It's a side effect of the fragile kernel->user interface in perf and likely hard to fix.

The only thing I can do is to ignore failures here and only print messages.

-Andi

#22 lore raw
In reply to Andi Kleen

On Wed, Mar 05, 2014 at 11:49:30AM -0800, Andi Kleen wrote:

SNIP

Quoted 35 lines

Patches also available from
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc perf/json

Example output:

% perf download
Downloading models file
Downloading readme.txt

2014-03-05 10:39:33 URL:https://download.01.org/perfmon/readme.txt [10320/10320] -> "readme.txt" [1]
2014-03-05 10:39:34 URL:https://download.01.org/perfmon/mapfile.csv [1207/1207] -> "mapfile.csv" [1]

Downloading events file
% perf list
...
br_inst_exec.all_branches [Speculative and retired
branches]
br_inst_exec.all_conditional [Speculative and retired
macro-conditional branches]
br_inst_exec.all_direct_jmp [Speculative and retired
macro-unconditional branches excluding calls and indirects]
... 333 more new events ...

% perf stat -e br_inst_exec.all_direct_jmp true

Performance counter stats for 'true':

             6,817      cpu/br_inst_exec.all_direct_jmp/                                   

       0.003503212 seconds time elapsed

One nice feature is that a pointer to the specification update is now included in the description, which will hopefully clear up many problems:

Quoted 11 lines

% perf list
...
mem_load_uops_l3_hit_retired.xsnp_hit [Retired load uops which
data sources were L3 and cross-core snoop hits in on-pkg core cache. Supports address when precise. Spec
update: HSM26, HSM30 (Precise event)]
...

nice! going to review this soon

jirka

#23 lore raw
In reply to Andi Kleen

On Wed, 2014-03-05 at 11:49 -0800, Andi Kleen wrote:
[...]

Quoted 2 lines

I implemented an automatic downloader to get the event file for the current CPU. The events are stored in ~/.events.

[...]

This directory should be placed under $XDG_CACHE_DIR (default:
~/.cache), and should have a more specific name like 'perf-events'.

It might be useful to also support a system directory for these, under /var/cache.

Ben.

Signature · 2 lines
Ben Hutchings
Computers are not intelligent.	They only think they are.
author
#24 lore raw
Quoted 8 lines · Ben Hutchings

On Wed, 2014-03-05 at 11:49 -0800, Andi Kleen wrote:
[...]

Quoted 2 lines

I implemented an automatic downloader to get the event file for the current CPU. The events are stored in ~/.events.

[...]

This directory should be placed under $XDG_CACHE_DIR (default:
~/.cache), and should have a more specific name like 'perf-events'.

Ok.

I don't like calling it perf-events because it will be used by other non perf tools too. I guess pmu-events could do.

Quoted 3 lines

It might be useful to also support a system directory for these, under /var/cache.

I don't want people shipping those. It just causes lots of outdated events everywhere, like in oprofile.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#25 lore raw
Quoted 16 lines · Andi Kleen
Quoted 8 lines · Ben Hutchings

On Wed, 2014-03-05 at 11:49 -0800, Andi Kleen wrote:
[...]

Quoted 2 lines

I implemented an automatic downloader to get the event file for the current CPU. The events are stored in ~/.events.

[...]

This directory should be placed under $XDG_CACHE_DIR (default:
~/.cache), and should have a more specific name like 'perf-events'.

Ok.

I don't like calling it perf-events because it will be used by other non perf tools too. I guess pmu-events could do.

Quoted 1 line
Quoted 5 lines
Quoted 2 lines

It might be useful to also support a system directory for these, under /var/cache.

I don't want people shipping those. It just causes lots of outdated events everywhere, like in oprofile.

Sure, no-one should be shipping files under /var/cache. It would be a an alternate download location.

Ben.

Signature · 2 lines
Ben Hutchings
Computers are not intelligent.	They only think they are.
author
#26 lore raw
Quoted 5 lines
Quoted 2 lines

I don't want people shipping those. It just causes lots of outdated events everywhere, like in oprofile.

Sure, no-one should be shipping files under /var/cache. It would be a an alternate download location.

You mean root perf download downloading there? But then people would download as root. I think I prefer them downloading as their own user.

So I think it's better to have it per user.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.