Skip to contentSkip to conversation

perf/core improvements and fixes

GIT PULL New Arnaldo Carvalho de Melo 2 participants 12 messages Feb 16, 2016 – Feb 17, 2016 perf events

lore
Share & tools
lore

Arnaldo Carvalho de Melo

author maintainer
#0 lore raw

Hi Ingo,

Please consider pulling,

- Arnaldo

The following changes since commit fe7a2eaa71c55aadbf95d01d32df8dccc0db0646:

Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-02-16 08:45:56 +0100)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo

for you to fetch changes up to cb110f471025f3278978aaccb18f3164ea2b8189:

perf stat: Move noise/running printing into printout (2016-02-16 17:13:02 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

User visible:

- Make 'perf record' collect CPU cache info in the perf.data file header:

  $ perf record usleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.017 MB perf.data (7 samples) ]
  $ perf report --header-only -I | tail -10 | head -8
  # CPU cache info:
  #  L1 Data                 32K [0-1]
  #  L1 Instruction          32K [0-1]
  #  L1 Data                 32K [2-3]
  #  L1 Instruction          32K [2-3]
  #  L2 Unified             256K [0-1]
  #  L2 Unified             256K [2-3]
  #  L3 Unified            4096K [0-3]
  $

Will be used in 'perf c2c' and eventually in 'perf diff' to allow, for instance running the same workload in multiple machines and then when using 'diff' show the hardware difference. (Jiri Olsa)

- 'perf stat' now shows shadow metrics (insn per cycle, etc) in interval mode too. E.g:

    # perf stat -I 1000 -e instructions,cycles sleep 1
    #         time   counts unit events
       1.000215928  519,620      instructions     #  0.69 insn per cycle
       1.000215928  752,003      cycles
    <SNIP>

Infrastructure:

- libapi now can also use pr_{warning,info,debug}() and that can be set by tools using it (Jiri Olsa)

- libapi adopts filename__read_str() from perf, adds sysfs__read_str() (Jiri Olsa)

- Add check for java alternatives cmd in jvmti Makefile, so that it manages to automatically find the right path for the JDK devel files in Ubuntu like systems in addition to Fedora like ones (Stephane Eranian)

----------------------------------------------------------------
Andi Kleen (3):
perf stat: Abstract stat metrics printing perf stat: Add support for metrics in interval mode perf stat: Move noise/running printing into printout

Arnaldo Carvalho de Melo (1):
perf debug: Rename __eprintf(va_list args) to veprintf

Jiri Olsa (5):
tools lib api: Add debug output support tools lib api fs: Adopt filename__read_str from perf tools lib api fs: Add sysfs__read_str function perf tools: Initialize libapi debug output perf tools: Add perf data cache feature

Stephane Eranian (1):
perf jvmti: Add check for java alternatives cmd in Makefile

 tools/lib/api/Build            |   1 +
 tools/lib/api/Makefile         |   1 +
 tools/lib/api/debug-internal.h |  20 +++
 tools/lib/api/debug.c          |  28 +++++
 tools/lib/api/debug.h          |  10 ++
 tools/lib/api/fs/fs.c          |  64 ++++++++++
 tools/lib/api/fs/fs.h          |   3 +
 tools/perf/builtin-stat.c      | 202 +++++++++++++++---------------
 tools/perf/jvmti/Makefile      |   6 +-
 tools/perf/perf.c              |   2 +
 tools/perf/util/debug.c        |  36 ++++--
 tools/perf/util/debug.h        |   1 +
 tools/perf/util/env.c          |  13 ++
 tools/perf/util/env.h          |  15 +++
 tools/perf/util/header.c       | 270 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/header.h       |   1 +
 tools/perf/util/stat-shadow.c  | 211 +++++++++++++++++---------------
 tools/perf/util/stat.h         |  15 ++-
 tools/perf/util/trace-event.c  |   1 +
 tools/perf/util/util.c         |  48 --------
 tools/perf/util/util.h         |   1 -
 21 files changed, 694 insertions(+), 255 deletions(-)
 create mode 100644 tools/lib/api/debug-internal.h
 create mode 100644 tools/lib/api/debug.c
 create mode 100644 tools/lib/api/debug.h
author maintainer
#1 lore raw

PATCH 1/10 perf jvmti: Add check for java alternatives cmd in Makefile

From: Stephane Eranian <[email protected]>

This patch modifies the jvmti makefile to check if the /usr/sbin/java-update-alternatives utility is present. If so, then use it, if not then use the altenatives command.

This helps handle the difference between Ubuntu and Fedora Linux distributions.

---
 tools/perf/jvmti/Makefile | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/perf/jvmti/Makefile +5 -1
index 5968f8332a28..0277a64b391b 100644
@@ -35,8 +35,12 @@ SOLIBEXT=so
 
 # The following works at least on fedora 23, you may need the next
 # line for other distros.
+ifeq (,$(wildcard /usr/sbin/update-java-alternatives))
 JDIR=$(shell alternatives --display java | tail -1 | cut -d' ' -f 5 | sed 's%/jre/bin/java.%%g')
-#JDIR=$(shell /usr/sbin/update-java-alternatives -l | head -1 | cut -d ' ' -f 3)
+else
+JDIR=$(shell /usr/sbin/update-java-alternatives -l | head -1 | cut -d ' ' -f 3)
+endif
+
 # -lrt required in 32-bit mode for clock_gettime()
 LIBS=-lelf -lrt
 INCDIR=-I $(JDIR)/include -I $(JDIR)/include/linux
Signature · 1 line
2.5.0
author maintainer
#2 lore raw

PATCH 2/10 tools lib api: Add debug output support

In reply to Arnaldo Carvalho de Melo

From: Jiri Olsa <[email protected]>

Adding support for warning/info/debug output within libapi code. Adding following macros:

pr_warning(fmt, ...)
pr_info(fmt, ...)
pr_debug(fmt, ...)

Also adding libapi_set_print function to set above functions. This will be used in perf to set standard debug handlers for libapi.

Adding 2 header files:
debug.h
- to be used outside libapi, contains
libapi_set_print interface

debug-internal.h
- to be used within libapi, contains
pr_warning/pr_info/pr_debug definitions

---
 tools/lib/api/Build            |  1 +
 tools/lib/api/Makefile         |  1 +
 tools/lib/api/debug-internal.h | 20 ++++++++++++++++++++
 tools/lib/api/debug.c          | 28 ++++++++++++++++++++++++++++
 tools/lib/api/debug.h          | 10 ++++++++++
 5 files changed, 60 insertions(+)
 create mode 100644 tools/lib/api/debug-internal.h
 create mode 100644 tools/lib/api/debug.c
 create mode 100644 tools/lib/api/debug.h
tools/lib/api/Build +1 -0
index e8b8a23b9bf4..954c644f7ad9 100644
@@ -1,3 +1,4 @@
 libapi-y += fd/
 libapi-y += fs/
 libapi-y += cpu.o
+libapi-y += debug.o
tools/lib/api/Makefile +1 -0
index d85904dc9b38..bbc82c614bee 100644
@@ -18,6 +18,7 @@ LIBFILE = $(OUTPUT)libapi.a
 CFLAGS := $(EXTRA_WARNINGS) $(EXTRA_CFLAGS)
 CFLAGS += -ggdb3 -Wall -Wextra -std=gnu99 -Werror -O6 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fPIC
 CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
+CFLAGS += -I$(srctree)/tools/lib/api
 
 RM = rm -f
 
tools/lib/api/debug-internal.h new file +20 -0
new file mode 100644
index 000000000000..188f7880eafe
@@ -0,0 +1,20 @@
+#ifndef __API_DEBUG_INTERNAL_H__
+#define __API_DEBUG_INTERNAL_H__
+
+#include "debug.h"
+
+#define __pr(func, fmt, ...)	\
+do {				\
+	if ((func))		\
+		(func)("libapi: " fmt, ##__VA_ARGS__); \
+} while (0)
+
+extern libapi_print_fn_t __pr_warning;
+extern libapi_print_fn_t __pr_info;
+extern libapi_print_fn_t __pr_debug;
+
+#define pr_warning(fmt, ...)	__pr(__pr_warning, fmt, ##__VA_ARGS__)
+#define pr_info(fmt, ...)	__pr(__pr_info, fmt, ##__VA_ARGS__)
+#define pr_debug(fmt, ...)	__pr(__pr_debug, fmt, ##__VA_ARGS__)
+
+#endif /* __API_DEBUG_INTERNAL_H__ */
tools/lib/api/debug.c new file +28 -0
new file mode 100644
index 000000000000..5fa5cf500a1f
@@ -0,0 +1,28 @@
+#include <stdio.h>
+#include <stdarg.h>
+#include "debug.h"
+#include "debug-internal.h"
+
+static int __base_pr(const char *format, ...)
+{
+	va_list args;
+	int err;
+
+	va_start(args, format);
+	err = vfprintf(stderr, format, args);
+	va_end(args);
+	return err;
+}
+
+libapi_print_fn_t __pr_warning = __base_pr;
+libapi_print_fn_t __pr_info    = __base_pr;
+libapi_print_fn_t __pr_debug;
+
+void libapi_set_print(libapi_print_fn_t warn,
+		      libapi_print_fn_t info,
+		      libapi_print_fn_t debug)
+{
+	__pr_warning = warn;
+	__pr_info    = info;
+	__pr_debug   = debug;
+}
tools/lib/api/debug.h new file +10 -0
new file mode 100644
index 000000000000..a0872f68fc56
@@ -0,0 +1,10 @@
+#ifndef __API_DEBUG_H__
+#define __API_DEBUG_H__
+
+typedef int (*libapi_print_fn_t)(const char *, ...);
+
+void libapi_set_print(libapi_print_fn_t warn,
+		      libapi_print_fn_t info,
+		      libapi_print_fn_t debug);
+
+#endif /* __API_DEBUG_H__ */
Signature · 1 line
2.5.0
author maintainer
#3 lore raw

PATCH 3/10 tools lib api fs: Adopt filename__read_str from perf

In reply to Arnaldo Carvalho de Melo

From: Jiri Olsa <[email protected]>

We already moved similar functions in here, also it'll be useful for sysfs__read_str addition in following patch.

---
 tools/lib/api/fs/fs.c         | 51 +++++++++++++++++++++++++++++++++++++++++++
 tools/lib/api/fs/fs.h         |  2 ++
 tools/perf/util/trace-event.c |  1 +
 tools/perf/util/util.c        | 48 ----------------------------------------
 tools/perf/util/util.h        |  1 -
 5 files changed, 54 insertions(+), 49 deletions(-)
tools/lib/api/fs/fs.c +51 -0
index 459599d1b6c4..2cbf6773ca5d 100644
@@ -13,6 +13,7 @@
 #include <sys/mount.h>
 
 #include "fs.h"
+#include "debug-internal.h"
 
 #define _STR(x) #x
 #define STR(x) _STR(x)
@@ -300,6 +301,56 @@ int filename__read_ull(const char *filename, unsigned long long *value)
 	return err;
 }
 
+#define STRERR_BUFSIZE  128     /* For the buffer size of strerror_r */
+
+int filename__read_str(const char *filename, char **buf, size_t *sizep)
+{
+	size_t size = 0, alloc_size = 0;
+	void *bf = NULL, *nbf;
+	int fd, n, err = 0;
+	char sbuf[STRERR_BUFSIZE];
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return -errno;
+
+	do {
+		if (size == alloc_size) {
+			alloc_size += BUFSIZ;
+			nbf = realloc(bf, alloc_size);
+			if (!nbf) {
+				err = -ENOMEM;
+				break;
+			}
+
+			bf = nbf;
+		}
+
+		n = read(fd, bf + size, alloc_size - size);
+		if (n < 0) {
+			if (size) {
+				pr_warning("read failed %d: %s\n", errno,
+					 strerror_r(errno, sbuf, sizeof(sbuf)));
+				err = 0;
+			} else
+				err = -errno;
+
+			break;
+		}
+
+		size += n;
+	} while (n > 0);
+
+	if (!err) {
+		*sizep = size;
+		*buf   = bf;
+	} else
+		free(bf);
+
+	close(fd);
+	return err;
+}
+
 int sysfs__read_ull(const char *entry, unsigned long long *value)
 {
 	char path[PATH_MAX];
tools/lib/api/fs/fs.h +2 -0
index d024a7f682f6..858922b61141 100644
@@ -2,6 +2,7 @@
 #define __API_FS__
 
 #include <stdbool.h>
+#include <unistd.h>
 
 /*
  * On most systems <limits.h> would have given us this, but  not on some systems
@@ -26,6 +27,7 @@ FS(tracefs)
 
 int filename__read_int(const char *filename, int *value);
 int filename__read_ull(const char *filename, unsigned long long *value);
+int filename__read_str(const char *filename, char **buf, size_t *sizep);
 
 int sysctl__read_int(const char *sysctl, int *value);
 int sysfs__read_int(const char *entry, int *value);
tools/perf/util/trace-event.c +1 -0
index 802bb868d446..8ae051e0ec79 100644
@@ -10,6 +10,7 @@
 #include <linux/err.h>
 #include <traceevent/event-parse.h>
 #include <api/fs/tracing_path.h>
+#include <api/fs/fs.h>
 #include "trace-event.h"
 #include "machine.h"
 #include "util.h"
tools/perf/util/util.c +0 -48

54 lines not on this page · raw message

tools/perf/util/util.h +0 -1
index a8615816a00d..3dd04089e8be 100644
@@ -303,7 +303,6 @@ char *__get_srcline(struct dso *dso, u64 addr, struct symbol *sym,
 		  bool show_sym, bool unwind_inlines);
 void free_srcline(char *srcline);
 
-int filename__read_str(const char *filename, char **buf, size_t *sizep);
 int perf_event_paranoid(void);
 
 void mem_bswap_64(void *src, int byte_size);
Signature · 1 line
2.5.0
author maintainer
#4 lore raw

PATCH 4/10 tools lib api fs: Add sysfs__read_str function

In reply to Arnaldo Carvalho de Melo

From: Jiri Olsa <[email protected]>

Adding sysfs__read_str function to ease up reading string files from sysfs. New interface is:

  int sysfs__read_str(const char *entry, char **buf, size_t *sizep);
---
 tools/lib/api/fs/fs.c | 13 +++++++++++++
 tools/lib/api/fs/fs.h |  1 +
 2 files changed, 14 insertions(+)
tools/lib/api/fs/fs.c +13 -0
index 2cbf6773ca5d..ef78c22ff44d 100644
@@ -377,6 +377,19 @@ int sysfs__read_int(const char *entry, int *value)
 	return filename__read_int(path, value);
 }
 
+int sysfs__read_str(const char *entry, char **buf, size_t *sizep)
+{
+	char path[PATH_MAX];
+	const char *sysfs = sysfs__mountpoint();
+
+	if (!sysfs)
+		return -1;
+
+	snprintf(path, sizeof(path), "%s/%s", sysfs, entry);
+
+	return filename__read_str(path, buf, sizep);
+}
+
 int sysctl__read_int(const char *sysctl, int *value)
 {
 	char path[PATH_MAX];
tools/lib/api/fs/fs.h +1 -0
index 858922b61141..9f6598098dc5 100644
@@ -32,4 +32,5 @@ int filename__read_str(const char *filename, char **buf, size_t *sizep);
 int sysctl__read_int(const char *sysctl, int *value);
 int sysfs__read_int(const char *entry, int *value);
 int sysfs__read_ull(const char *entry, unsigned long long *value);
+int sysfs__read_str(const char *entry, char **buf, size_t *sizep);
 #endif /* __API_FS__ */
Signature · 1 line
2.5.0
author maintainer
#5 lore raw

PATCH 5/10 perf debug: Rename __eprintf(va_list args) to veprintf

In reply to Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

Adhering to the naming convention used when va_args is in a printf like function, e.g. stdio.h.

---
 tools/perf/util/debug.c | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)
tools/perf/util/debug.c +5 -10
index 86d9c7302598..d6c8d2b2cd70 100644
@@ -22,7 +22,7 @@ int debug_ordered_events;
 static int redirect_to_stderr;
 int debug_data_convert;
 
-static int _eprintf(int level, int var, const char *fmt, va_list args)
+int veprintf(int level, int var, const char *fmt, va_list args)
 {
 	int ret = 0;
 
@@ -36,24 +36,19 @@ static int _eprintf(int level, int var, const char *fmt, va_list args)
 	return ret;
 }
 
-int veprintf(int level, int var, const char *fmt, va_list args)
-{
-	return _eprintf(level, var, fmt, args);
-}
-
 int eprintf(int level, int var, const char *fmt, ...)
 {
 	va_list args;
 	int ret;
 
 	va_start(args, fmt);
-	ret = _eprintf(level, var, fmt, args);
+	ret = veprintf(level, var, fmt, args);
 	va_end(args);
 
 	return ret;
 }
 
-static int __eprintf_time(u64 t, const char *fmt, va_list args)
+static int veprintf_time(u64 t, const char *fmt, va_list args)
 {
 	int ret = 0;
 	u64 secs, usecs, nsecs = t;
@@ -75,7 +70,7 @@ int eprintf_time(int level, int var, u64 t, const char *fmt, ...)
 
 	if (var >= level) {
 		va_start(args, fmt);
-		ret = __eprintf_time(t, fmt, args);
+		ret = veprintf_time(t, fmt, args);
 		va_end(args);
 	}
 
@@ -91,7 +86,7 @@ void pr_stat(const char *fmt, ...)
 	va_list args;
 
 	va_start(args, fmt);
-	_eprintf(1, verbose, fmt, args);
+	veprintf(1, verbose, fmt, args);
 	va_end(args);
 	eprintf(1, verbose, "\n");
 }
Signature · 1 line
2.5.0
author maintainer
#6 lore raw

PATCH 6/10 perf tools: Initialize libapi debug output

In reply to Arnaldo Carvalho de Melo

From: Jiri Olsa <[email protected]>

Setting libapi debug output functions to use perf functions.

---
 tools/perf/perf.c       |  2 ++
 tools/perf/util/debug.c | 21 +++++++++++++++++++++
 tools/perf/util/debug.h |  1 +
 3 files changed, 24 insertions(+)
tools/perf/perf.c +2 -0
index a929618b8eb6..144047c396f0 100644
@@ -613,6 +613,8 @@ int main(int argc, const char **argv)
 	 */
 	pthread__block_sigwinch();
 
+	perf_debug_setup();
+
 	while (1) {
 		static int done_help;
 		int was_alias = run_argv(&argc, &argv);
tools/perf/util/debug.c +21 -0
index d6c8d2b2cd70..ff7e86ad1b06 100644
@@ -5,6 +5,7 @@
 #include <string.h>
 #include <stdarg.h>
 #include <stdio.h>
+#include <api/debug.h>
 
 #include "cache.h"
 #include "color.h"
@@ -187,3 +188,23 @@ int perf_debug_option(const char *str)
 	free(s);
 	return 0;
 }
+
+#define DEBUG_WRAPPER(__n, __l)				\
+static int pr_ ## __n ## _wrapper(const char *fmt, ...)	\
+{							\
+	va_list args;					\
+	int ret;					\
+							\
+	va_start(args, fmt);				\
+	ret = veprintf(__l, verbose, fmt, args);	\
+	va_end(args);					\
+	return ret;					\
+}
+
+DEBUG_WRAPPER(warning, 0);
+DEBUG_WRAPPER(debug, 1);
+
+void perf_debug_setup(void)
+{
+	libapi_set_print(pr_warning_wrapper, pr_warning_wrapper, pr_debug_wrapper);
+}
tools/perf/util/debug.h +1 -0
index 8b9a088c32ab..14bafda79eda 100644
@@ -53,5 +53,6 @@ int eprintf_time(int level, int var, u64 t, const char *fmt, ...) __attribute__(
 int veprintf(int level, int var, const char *fmt, va_list args);
 
 int perf_debug_option(const char *str);
+void perf_debug_setup(void);
 
 #endif	/* __PERF_DEBUG_H */
Signature · 1 line
2.5.0
author maintainer
#7 lore raw

PATCH 7/10 perf tools: Add perf data cache feature

In reply to Arnaldo Carvalho de Melo

From: Jiri Olsa <[email protected]>

Storing CPU cache details under perf data. It's stored as new HEADER_CACHE feature and it's displayed under header info with -I option:

  $ perf report --header-only -I
  ...
  # CPU cache info:
  #  L1 Data                 32K [0-1]
  #  L1 Instruction          32K [0-1]
  #  L1 Data                 32K [2-3]
  #  L1 Instruction          32K [2-3]
  #  L2 Unified             256K [0-1]
  #  L2 Unified             256K [2-3]
  #  L3 Unified            4096K [0-3]
  ...

All distinct caches are stored/displayed.

Cc David Ahern <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Peter Zijlstra <[email protected]>
[ Fixed leak on process_caches(), s/cache_level/cpu_cache_level/g ]
---
 tools/perf/util/env.c    |  13 +++
 tools/perf/util/env.h    |  15 +++
 tools/perf/util/header.c | 270 +++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/header.h |   1 +
 4 files changed, 299 insertions(+)
tools/perf/util/env.c +13 -0
index 7dd5939dea2e..49a11d9d8b8f 100644
@@ -6,6 +6,8 @@ struct perf_env perf_env;
 
 void perf_env__exit(struct perf_env *env)
 {
+	int i;
+
 	zfree(&env->hostname);
 	zfree(&env->os_release);
 	zfree(&env->version);
@@ -19,6 +21,10 @@ void perf_env__exit(struct perf_env *env)
 	zfree(&env->numa_nodes);
 	zfree(&env->pmu_mappings);
 	zfree(&env->cpu);
+
+	for (i = 0; i < env->caches_cnt; i++)
+		cpu_cache_level__free(&env->caches[i]);
+	zfree(&env->caches);
 }
 
 int perf_env__set_cmdline(struct perf_env *env, int argc, const char *argv[])
@@ -75,3 +81,10 @@ int perf_env__read_cpu_topology_map(struct perf_env *env)
 	env->nr_cpus_avail = nr_cpus;
 	return 0;
 }
+
+void cpu_cache_level__free(struct cpu_cache_level *cache)
+{
+	free(cache->type);
+	free(cache->map);
+	free(cache->size);
+}
tools/perf/util/env.h +15 -0
index 0132b9557c02..56cffb60a0b4 100644
@@ -1,11 +1,23 @@
 #ifndef __PERF_ENV_H
 #define __PERF_ENV_H
 
+#include <linux/types.h>
+
 struct cpu_topology_map {
 	int	socket_id;
 	int	core_id;
 };
 
+struct cpu_cache_level {
+	u32	level;
+	u32	line_size;
+	u32	sets;
+	u32	ways;
+	char	*type;
+	char	*size;
+	char	*map;
+};
+
 struct perf_env {
 	char			*hostname;
 	char			*os_release;
@@ -31,6 +43,8 @@ struct perf_env {
 	char			*numa_nodes;
 	char			*pmu_mappings;
 	struct cpu_topology_map	*cpu;
+	struct cpu_cache_level	*caches;
+	int			 caches_cnt;
 };
 
 extern struct perf_env perf_env;
@@ -41,4 +55,5 @@ int perf_env__set_cmdline(struct perf_env *env, int argc, const char *argv[]);
 
 int perf_env__read_cpu_topology_map(struct perf_env *env);
 
+void cpu_cache_level__free(struct cpu_cache_level *cache);
 #endif /* __PERF_ENV_H */
tools/perf/util/header.c +270 -0

300 lines not on this page · raw message

tools/perf/util/header.h +1 -0
index cff9892452ee..3d87ca823c0a 100644
@@ -32,6 +32,7 @@ enum {
 	HEADER_GROUP_DESC,
 	HEADER_AUXTRACE,
 	HEADER_STAT,
+	HEADER_CACHE,
 	HEADER_LAST_FEATURE,
 	HEADER_FEAT_BITS	= 256,
 };
Signature · 1 line
2.5.0
author maintainer
#8 lore raw

PATCH 8/10 perf stat: Abstract stat metrics printing

In reply to Arnaldo Carvalho de Melo

From: Andi Kleen <[email protected]>

Abstract the printing of shadow metrics. Instead of every metric calling fprintf directly and taking care of indentation, use two call backs: one to print metrics and another to start a new line.

This will allow adding metrics to CSV mode and also using them for other purposes.

The computation of padding is now done in the central callback, instead of every metric doing it manually. This makes it easier to add new metrics.

v2: Refactor functions, printout now does more. Move shadow printing. Improve fallback callbacks. Don't
use void * callback data.
v3: Remove unnecessary hunk. Add typedef for new_line
v4: Remove unnecessary hunk. Don't print metrics for CSV/interval mode yet. Move printout change to separate patch.
v5: Fix bisect bugs. Avoid bogus frontend cycles printing. Fix indentation in different aggregation modes.
v6: Delay newline handling

---
 tools/perf/builtin-stat.c     |  71 ++++++++++++--
 tools/perf/util/stat-shadow.c | 211 +++++++++++++++++++++++-------------------
 tools/perf/util/stat.h        |  15 ++-
 3 files changed, 194 insertions(+), 103 deletions(-)
tools/perf/builtin-stat.c +65 -6
index 038e877081b6..fabcadba1f19 100644
@@ -735,6 +735,58 @@ static void aggr_printout(struct perf_evsel *evsel, int id, int nr)
 	}
 }
 
+struct outstate {
+	FILE *fh;
+	bool newline;
+};
+
+#define METRIC_LEN  35
+
+static void new_line_std(void *ctx)
+{
+	struct outstate *os = ctx;
+
+	os->newline = true;
+}
+
+static void do_new_line_std(struct outstate *os)
+{
+	fputc('\n', os->fh);
+	if (stat_config.aggr_mode == AGGR_NONE)
+		fprintf(os->fh, "        ");
+	if (stat_config.aggr_mode == AGGR_CORE)
+		fprintf(os->fh, "                  ");
+	if (stat_config.aggr_mode == AGGR_SOCKET)
+		fprintf(os->fh, "            ");
+	fprintf(os->fh, "                                                 ");
+}
+
+static void print_metric_std(void *ctx, const char *color, const char *fmt,
+			     const char *unit, double val)
+{
+	struct outstate *os = ctx;
+	FILE *out = os->fh;
+	int n;
+	bool newline = os->newline;
+
+	os->newline = false;
+
+	if (unit == NULL || fmt == NULL) {
+		fprintf(out, "%-*s", METRIC_LEN, "");
+		return;
+	}
+
+	if (newline)
+		do_new_line_std(os);
+
+	n = fprintf(out, " # ");
+	if (color)
+		n += color_fprintf(out, color, fmt, val);
+	else
+		n += fprintf(out, fmt, val);
+	fprintf(out, " %-*s", METRIC_LEN - n - 1, unit);
+}
+
 static void nsec_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 {
 	FILE *output = stat_config.output;
@@ -795,20 +847,27 @@ static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 
 static void printout(int id, int nr, struct perf_evsel *counter, double uval)
 {
-	int cpu = cpu_map__id_to_cpu(id);
+	struct outstate os = { .fh = stat_config.output };
+	struct perf_stat_output_ctx out;
+	print_metric_t pm = print_metric_std;
+	void (*nl)(void *);
 
-	if (stat_config.aggr_mode == AGGR_GLOBAL)
-		cpu = 0;
+	nl = new_line_std;
 
 	if (nsec_counter(counter))
 		nsec_printout(id, nr, counter, uval);
 	else
 		abs_printout(id, nr, counter, uval);
 
+	out.print_metric = pm;
+	out.new_line = nl;
+	out.ctx = &os;
+
 	if (!csv_output && !stat_config.interval)
-		perf_stat__print_shadow_stats(stat_config.output, counter,
-					      uval, cpu,
-					      stat_config.aggr_mode);
+		perf_stat__print_shadow_stats(counter, uval,
+				stat_config.aggr_mode == AGGR_GLOBAL ? 0 :
+				cpu_map__id_to_cpu(id),
+				&out);
 }
 
 static void print_aggr(char *prefix)
tools/perf/util/stat-shadow.c +116 -95

369 lines not on this page · raw message

tools/perf/util/stat.h +13 -2
index 2af63c9cb59f..f02af68adc04 100644
@@ -68,11 +68,22 @@ void perf_stat_evsel_id_init(struct perf_evsel *evsel);
 
 extern struct stats walltime_nsecs_stats;
 
+typedef void (*print_metric_t)(void *ctx, const char *color, const char *unit,
+			       const char *fmt, double val);
+typedef void (*new_line_t )(void *ctx);
+
 void perf_stat__reset_shadow_stats(void);
 void perf_stat__update_shadow_stats(struct perf_evsel *counter, u64 *count,
 				    int cpu);
-void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
-				   double avg, int cpu, enum aggr_mode aggr);
+struct perf_stat_output_ctx {
+	void *ctx;
+	print_metric_t print_metric;
+	new_line_t new_line;
+};
+
+void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
+				   double avg, int cpu,
+				   struct perf_stat_output_ctx *out);
 
 int perf_evlist__alloc_stats(struct perf_evlist *evlist, bool alloc_raw);
 void perf_evlist__free_stats(struct perf_evlist *evlist);
Signature · 1 line
2.5.0
author maintainer
#9 lore raw

PATCH 9/10 perf stat: Add support for metrics in interval mode

In reply to Arnaldo Carvalho de Melo

From: Andi Kleen <[email protected]>

Now that we can modify the metrics printout functions easily, it's straight forward to support metric printing for interval mode. All that is needed is to print the time stamp on every new line. Pass the prefix into the context and print it out.

v2: Move wrong hunk to here.

Committer note:

Before:

[root@jouet ~]# perf stat -I 1000 -e instructions,cycles sleep 1
# time counts unit events

       1.000168216   538,913      instructions
       1.000168216   748,765      cycles
       1.000660048   153,741      instructions
       1.000660048   214,066      cycles

After:

  # perf stat -I 1000 -e instructions,cycles sleep 1
  #           time    counts unit events
       1.000215928   519,620      instructions              #    0.69  insn per cycle
       1.000215928   752,003      cycles
       1.000946033   148,502      instructions              #    0.33  insn per cycle
       1.000946033   160,104      cycles
---
 tools/perf/builtin-stat.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
tools/perf/builtin-stat.c +13 -7
index fabcadba1f19..5710bdb058d2 100644
@@ -738,6 +738,7 @@ static void aggr_printout(struct perf_evsel *evsel, int id, int nr)
 struct outstate {
 	FILE *fh;
 	bool newline;
+	const char *prefix;
 };
 
 #define METRIC_LEN  35
@@ -752,6 +753,7 @@ static void new_line_std(void *ctx)
 static void do_new_line_std(struct outstate *os)
 {
 	fputc('\n', os->fh);
+	fputs(os->prefix, os->fh);
 	if (stat_config.aggr_mode == AGGR_NONE)
 		fprintf(os->fh, "        ");
 	if (stat_config.aggr_mode == AGGR_CORE)
@@ -845,10 +847,14 @@ static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
 }
 
-static void printout(int id, int nr, struct perf_evsel *counter, double uval)
+static void printout(int id, int nr, struct perf_evsel *counter, double uval,
+		     char *prefix)
 {
-	struct outstate os = { .fh = stat_config.output };
 	struct perf_stat_output_ctx out;
+	struct outstate os = {
+		.fh = stat_config.output,
+		.prefix = prefix ? prefix : ""
+	};
 	print_metric_t pm = print_metric_std;
 	void (*nl)(void *);
 
@@ -863,7 +869,7 @@ static void printout(int id, int nr, struct perf_evsel *counter, double uval)
 	out.new_line = nl;
 	out.ctx = &os;
 
-	if (!csv_output && !stat_config.interval)
+	if (!csv_output)
 		perf_stat__print_shadow_stats(counter, uval,
 				stat_config.aggr_mode == AGGR_GLOBAL ? 0 :
 				cpu_map__id_to_cpu(id),
@@ -923,7 +929,7 @@ static void print_aggr(char *prefix)
 				continue;
 			}
 			uval = val * counter->scale;
-			printout(id, nr, counter, uval);
+			printout(id, nr, counter, uval, prefix);
 			if (!csv_output)
 				print_noise(counter, 1.0);
 
@@ -954,7 +960,7 @@ static void print_aggr_thread(struct perf_evsel *counter, char *prefix)
 			fprintf(output, "%s", prefix);
 
 		uval = val * counter->scale;
-		printout(thread, 0, counter, uval);
+		printout(thread, 0, counter, uval, prefix);
 
 		if (!csv_output)
 			print_noise(counter, 1.0);
@@ -1004,7 +1010,7 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	}
 
 	uval = avg * counter->scale;
-	printout(-1, 0, counter, uval);
+	printout(-1, 0, counter, uval, prefix);
 
 	print_noise(counter, avg);
 
@@ -1057,7 +1063,7 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 		}
 
 		uval = val * counter->scale;
-		printout(cpu, 0, counter, uval);
+		printout(cpu, 0, counter, uval, prefix);
 		if (!csv_output)
 			print_noise(counter, 1.0);
 		print_running(run, ena);
Signature · 1 line
2.5.0
author maintainer
#10 lore raw

PATCH 10/10 perf stat: Move noise/running printing into printout

In reply to Arnaldo Carvalho de Melo

From: Andi Kleen <[email protected]>

Move the running/noise printing into printout to avoid duplicated code in the callers.

v2: Merged with other patches. Remove unnecessary hunk.
Readd hunk that ended in earlier patch.
v3: Fix noise/running output in CSV mode
v4: Merge with later patch that also moves not supported printing.

---
 tools/perf/builtin-stat.c | 123 ++++++++++++----------------------------------
 1 file changed, 32 insertions(+), 91 deletions(-)
tools/perf/builtin-stat.c +32 -91
index 5710bdb058d2..15e4fcf34e0c 100644
@@ -848,7 +848,7 @@ static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 }
 
 static void printout(int id, int nr, struct perf_evsel *counter, double uval,
-		     char *prefix)
+		     char *prefix, u64 run, u64 ena, double noise)
 {
 	struct perf_stat_output_ctx out;
 	struct outstate os = {
@@ -860,6 +860,30 @@ static void printout(int id, int nr, struct perf_evsel *counter, double uval,
 
 	nl = new_line_std;
 
+	if (run == 0 || ena == 0) {
+		aggr_printout(counter, id, nr);
+
+		fprintf(stat_config.output, "%*s%s",
+			csv_output ? 0 : 18,
+			counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
+			csv_sep);
+
+		fprintf(stat_config.output, "%-*s%s",
+			csv_output ? 0 : unit_width,
+			counter->unit, csv_sep);
+
+		fprintf(stat_config.output, "%*s",
+			csv_output ? 0 : -25,
+			perf_evsel__name(counter));
+
+		if (counter->cgrp)
+			fprintf(stat_config.output, "%s%s",
+				csv_sep, counter->cgrp->name);
+
+		print_running(run, ena);
+		return;
+	}
+
 	if (nsec_counter(counter))
 		nsec_printout(id, nr, counter, uval);
 	else
@@ -874,6 +898,9 @@ static void printout(int id, int nr, struct perf_evsel *counter, double uval,
 				stat_config.aggr_mode == AGGR_GLOBAL ? 0 :
 				cpu_map__id_to_cpu(id),
 				&out);
+
+	print_noise(counter, noise);
+	print_running(run, ena);
 }
 
 static void print_aggr(char *prefix)
@@ -904,36 +931,8 @@ static void print_aggr(char *prefix)
 			if (prefix)
 				fprintf(output, "%s", prefix);
 
-			if (run == 0 || ena == 0) {
-				aggr_printout(counter, id, nr);
-
-				fprintf(output, "%*s%s",
-					csv_output ? 0 : 18,
-					counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-					csv_sep);
-
-				fprintf(output, "%-*s%s",
-					csv_output ? 0 : unit_width,
-					counter->unit, csv_sep);
-
-				fprintf(output, "%*s",
-					csv_output ? 0 : -25,
-					perf_evsel__name(counter));
-
-				if (counter->cgrp)
-					fprintf(output, "%s%s",
-						csv_sep, counter->cgrp->name);
-
-				print_running(run, ena);
-				fputc('\n', output);
-				continue;
-			}
 			uval = val * counter->scale;
-			printout(id, nr, counter, uval, prefix);
-			if (!csv_output)
-				print_noise(counter, 1.0);
-
-			print_running(run, ena);
+			printout(id, nr, counter, uval, prefix, run, ena, 1.0);
 			fputc('\n', output);
 		}
 	}
@@ -960,12 +959,7 @@ static void print_aggr_thread(struct perf_evsel *counter, char *prefix)
 			fprintf(output, "%s", prefix);
 
 		uval = val * counter->scale;
-		printout(thread, 0, counter, uval, prefix);
-
-		if (!csv_output)
-			print_noise(counter, 1.0);
-
-		print_running(run, ena);
+		printout(thread, 0, counter, uval, prefix, run, ena, 1.0);
 		fputc('\n', output);
 	}
 }
@@ -979,7 +973,6 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	FILE *output = stat_config.output;
 	struct perf_stat_evsel *ps = counter->priv;
 	double avg = avg_stats(&ps->res_stats[0]);
-	int scaled = counter->counts->scaled;
 	double uval;
 	double avg_enabled, avg_running;
 
@@ -989,32 +982,8 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	if (prefix)
 		fprintf(output, "%s", prefix);
 
-	if (scaled == -1 || !counter->supported) {
-		fprintf(output, "%*s%s",
-			csv_output ? 0 : 18,
-			counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-			csv_sep);
-		fprintf(output, "%-*s%s",
-			csv_output ? 0 : unit_width,
-			counter->unit, csv_sep);
-		fprintf(output, "%*s",
-			csv_output ? 0 : -25,
-			perf_evsel__name(counter));
-
-		if (counter->cgrp)
-			fprintf(output, "%s%s", csv_sep, counter->cgrp->name);
-
-		print_running(avg_running, avg_enabled);
-		fputc('\n', output);
-		return;
-	}
-
 	uval = avg * counter->scale;
-	printout(-1, 0, counter, uval, prefix);
-
-	print_noise(counter, avg);
-
-	print_running(avg_running, avg_enabled);
+	printout(-1, 0, counter, uval, prefix, avg_running, avg_enabled, avg);
 	fprintf(output, "\n");
 }
 
@@ -1037,36 +1006,8 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 		if (prefix)
 			fprintf(output, "%s", prefix);
 
-		if (run == 0 || ena == 0) {
-			fprintf(output, "CPU%*d%s%*s%s",
-				csv_output ? 0 : -4,
-				perf_evsel__cpus(counter)->map[cpu], csv_sep,
-				csv_output ? 0 : 18,
-				counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-				csv_sep);
-
-				fprintf(output, "%-*s%s",
-					csv_output ? 0 : unit_width,
-					counter->unit, csv_sep);
-
-				fprintf(output, "%*s",
-					csv_output ? 0 : -25,
-					perf_evsel__name(counter));
-
-			if (counter->cgrp)
-				fprintf(output, "%s%s",
-					csv_sep, counter->cgrp->name);
-
-			print_running(run, ena);
-			fputc('\n', output);
-			continue;
-		}
-
 		uval = val * counter->scale;
-		printout(cpu, 0, counter, uval, prefix);
-		if (!csv_output)
-			print_noise(counter, 1.0);
-		print_running(run, ena);
+		printout(cpu, 0, counter, uval, prefix, run, ena, 1.0);
 
 		fputc('\n', output);
 	}
Signature · 1 line
2.5.0
#11 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 101 lines · Arnaldo Carvalho de Melo

Hi Ingo,

Please consider pulling,

- Arnaldo

The following changes since commit fe7a2eaa71c55aadbf95d01d32df8dccc0db0646:

Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-02-16 08:45:56 +0100)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo

for you to fetch changes up to cb110f471025f3278978aaccb18f3164ea2b8189:

perf stat: Move noise/running printing into printout (2016-02-16 17:13:02 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

User visible:

- Make 'perf record' collect CPU cache info in the perf.data file header:

  $ perf record usleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.017 MB perf.data (7 samples) ]
  $ perf report --header-only -I | tail -10 | head -8
  # CPU cache info:
  #  L1 Data                 32K [0-1]
  #  L1 Instruction          32K [0-1]
  #  L1 Data                 32K [2-3]
  #  L1 Instruction          32K [2-3]
  #  L2 Unified             256K [0-1]
  #  L2 Unified             256K [2-3]
  #  L3 Unified            4096K [0-3]
  $

Will be used in 'perf c2c' and eventually in 'perf diff' to allow, for instance running the same workload in multiple machines and then when using 'diff' show the hardware difference. (Jiri Olsa)

- 'perf stat' now shows shadow metrics (insn per cycle, etc) in interval mode too. E.g:

    # perf stat -I 1000 -e instructions,cycles sleep 1
    #         time   counts unit events
       1.000215928  519,620      instructions     #  0.69 insn per cycle
       1.000215928  752,003      cycles
    <SNIP>

Infrastructure:

- libapi now can also use pr_{warning,info,debug}() and that can be set by tools using it (Jiri Olsa)

- libapi adopts filename__read_str() from perf, adds sysfs__read_str() (Jiri Olsa)

- Add check for java alternatives cmd in jvmti Makefile, so that it manages to automatically find the right path for the JDK devel files in Ubuntu like systems in addition to Fedora like ones (Stephane Eranian)

----------------------------------------------------------------
Andi Kleen (3):
perf stat: Abstract stat metrics printing perf stat: Add support for metrics in interval mode perf stat: Move noise/running printing into printout

Arnaldo Carvalho de Melo (1):
perf debug: Rename __eprintf(va_list args) to veprintf

Jiri Olsa (5):
tools lib api: Add debug output support tools lib api fs: Adopt filename__read_str from perf tools lib api fs: Add sysfs__read_str function perf tools: Initialize libapi debug output perf tools: Add perf data cache feature

Stephane Eranian (1):
perf jvmti: Add check for java alternatives cmd in Makefile

 tools/lib/api/Build            |   1 +
 tools/lib/api/Makefile         |   1 +
 tools/lib/api/debug-internal.h |  20 +++
 tools/lib/api/debug.c          |  28 +++++
 tools/lib/api/debug.h          |  10 ++
 tools/lib/api/fs/fs.c          |  64 ++++++++++
 tools/lib/api/fs/fs.h          |   3 +
 tools/perf/builtin-stat.c      | 202 +++++++++++++++---------------
 tools/perf/jvmti/Makefile      |   6 +-
 tools/perf/perf.c              |   2 +
 tools/perf/util/debug.c        |  36 ++++--
 tools/perf/util/debug.h        |   1 +
 tools/perf/util/env.c          |  13 ++
 tools/perf/util/env.h          |  15 +++
 tools/perf/util/header.c       | 270 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/header.h       |   1 +
 tools/perf/util/stat-shadow.c  | 211 +++++++++++++++++---------------
Quoted 8 lines
 tools/perf/util/stat.h         |  15 ++-
 tools/perf/util/trace-event.c  |   1 +
 tools/perf/util/util.c         |  48 --------
 tools/perf/util/util.h         |   1 -
 21 files changed, 694 insertions(+), 255 deletions(-)
 create mode 100644 tools/lib/api/debug-internal.h
 create mode 100644 tools/lib/api/debug.c
 create mode 100644 tools/lib/api/debug.h

Pulled, thanks a lot Arnaldo!

Ingo