Skip to contentSkip to conversation

perf/core improvements and fixes

GIT PULL New Jiri Olsa 2 participants 11 messages Jun 3, 2014 – Jun 5, 2014 perf events

lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

hi Ingo,
please consider pulling

thanks,
jirka

The following changes since commit 9b261365dd73a5014b49033327ad881708e81f33:

Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf into perf/core (2014-06-03 20:22:40 +0200)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git tags/perf-core-for-mingo

for you to fetch changes up to fc9cabeabf42d76854059e7bce81a02645e7e5ca:

perf tools: Fix 'make help' message error (2014-06-03 21:35:12 +0200)

----------------------------------------------------------------
perf/core improvements and fixes:

. Warn the user when trace command is not available (Arnaldo Carvalho de Melo)

. Add warning when disabling perl scripting support due to missing devel files (Arnaldo Carvalho de Melo)

. Consider header files outside perf directory in tags target (Sebastian Andrzej Siewior)

. Allow overriding sysfs and proc finding with env var (Cody P Schafer)

. Fix "==" into "=" in ui_browser__warning assignment (zhangdianfang)

. Factor elide bool handling in sort code (Jiri Olsa)

. Fix poll return value propagation (Jiri Olsa)

. Fix 'make help' message error (Jianyu Zhan)

----------------------------------------------------------------
Arnaldo Carvalho de Melo (2):
perf trace: Warn the user when not available
perf tools: Add warning when disabling perl scripting support due to missing devel files

Cody P Schafer (1):
perf tools: Allow overriding sysfs and proc finding with env var

Jianyu Zhan (1):
perf tools: Fix 'make help' message error

Jiri Olsa (3):
perf tools: Remove elide setup for SORT_MODE__MEMORY mode perf tools: Move elide bool into perf_hpp_fmt struct perf record: Fix poll return value propagation

Sebastian Andrzej Siewior (1):
perf tools: Consider header files outside perf directory in tags target

zhangdianfang (1):

      perf tools: Fix "==" into "=" in ui_browser__warning assignment
 tools/lib/api/fs/fs.c          |  43 ++++++++++++++++-
 tools/perf/Makefile.perf       |  13 ++++--
 tools/perf/builtin-record.c    |   6 ++-
 tools/perf/config/Makefile     |   3 +-
 tools/perf/perf.c              |   8 +++-
 tools/perf/ui/browser.c        |   2 +-
 tools/perf/ui/browsers/hists.c |   8 ++--
 tools/perf/util/hist.h         |   8 +++-
 tools/perf/util/sort.c         | 103 ++++++++++++++++++++++-------------------
 tools/perf/util/sort.h         |   2 +-
 10 files changed, 132 insertions(+), 64 deletions(-)
author reviewer
#1 lore raw

PATCH 1/9 perf trace: Warn the user when not available

From: Arnaldo Carvalho de Melo <[email protected]>

When the audit-libs devel package is not found at build time we disable the 'trace' command, as we are not able to map syscall numbers to strings, but then the message the user is presented is cryptic:

[root@zoo linux]# trace ls
perf: 'ls' is not a perf-command. See 'perf --help'.

Fix it by presenting a more helpful message:

[root@zoo linux]# trace l
trace command not available: missing audit-libs devel package at build time.

Cc Adrian Hunter <[email protected]>
Cc David Ahern <[email protected]>
Cc Don Zickus <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 tools/perf/perf.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/perf/perf.c +6 -2
index 431798a..78f7b92 100644
@@ -481,14 +481,18 @@ int main(int argc, const char **argv)
 		fprintf(stderr, "cannot handle %s internally", cmd);
 		goto out;
 	}
-#ifdef HAVE_LIBAUDIT_SUPPORT
 	if (!prefixcmp(cmd, "trace")) {
+#ifdef HAVE_LIBAUDIT_SUPPORT
 		set_buildid_dir();
 		setup_path();
 		argv[0] = "trace";
 		return cmd_trace(argc, argv, NULL);
-	}
+#else
+		fprintf(stderr,
+			"trace command not available: missing audit-libs devel package at build time.\n");
+		goto out;
 #endif
+	}
 	/* Look for flags.. */
 	argv++;
 	argc--;
Signature · 1 line
1.8.3.1
author reviewer
#2 lore raw

PATCH 2/9 perf tools: Add warning when disabling perl scripting support due to missing devel files

In reply to Jiri Olsa

From: Arnaldo Carvalho de Melo <[email protected]>

We were just showing "libperl: OFF", unlike other features where we present the user with a message helping have a feature built in.

Fix it by adding the following message:

config/Makefile:450: Missing perl devel files. Disabling perl scripting support, consider installing perl-ExtUtils-Embed

Cc Adrian Hunter <[email protected]>
Cc David Ahern <[email protected]>
Cc Don Zickus <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 tools/perf/config/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/perf/config/Makefile +1 -0
index 729bbdf..319426f 100644
@@ -447,6 +447,7 @@ else
   ifneq ($(feature-libperl), 1)
     CFLAGS += -DNO_LIBPERL
     NO_LIBPERL := 1
+    msg := $(warning Missing perl devel files. Disabling perl scripting support, consider installing perl-ExtUtils-Embed);
   else
     LDFLAGS += $(PERL_EMBED_LDFLAGS)
     EXTLIBS += $(PERL_EMBED_LIBADD)
Signature · 1 line
1.8.3.1
author reviewer
#3 lore raw

PATCH 3/9 perf tools: Consider header files outside perf directory in tags target

In reply to Jiri Olsa

From: Sebastian Andrzej Siewior <[email protected]>

This fixes lookups like "vi -t event_format"

---
 tools/perf/Makefile.perf | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/perf/Makefile.perf +6 -3
index 67f7c05..6286e13 100644
@@ -815,17 +815,20 @@ INSTALL_DOC_TARGETS += quick-install-doc quick-install-man quick-install-html
 $(DOC_TARGETS):
 	$(QUIET_SUBDIR0)Documentation $(QUIET_SUBDIR1) $(@:doc=all)
 
+TAG_FOLDERS= . ../lib/traceevent ../lib/api ../lib/symbol
+TAG_FILES= ../../include/uapi/linux/perf_event.h
+
 TAGS:
 	$(RM) TAGS
-	$(FIND) . -name '*.[hcS]' -print | xargs etags -a
+	$(FIND) $(TAG_FOLDERS) -name '*.[hcS]' -print | xargs etags -a $(TAG_FILES)
 
 tags:
 	$(RM) tags
-	$(FIND) . -name '*.[hcS]' -print | xargs ctags -a
+	$(FIND) $(TAG_FOLDERS) -name '*.[hcS]' -print | xargs ctags -a $(TAG_FILES)
 
 cscope:
 	$(RM) cscope*
-	$(FIND) . -name '*.[hcS]' -print | xargs cscope -b
+	$(FIND) $(TAG_FOLDERS) -name '*.[hcS]' -print | xargs cscope -b $(TAG_FILES)
 
 ### Detect prefix changes
 TRACK_CFLAGS = $(subst ','\'',$(CFLAGS)):\
Signature · 1 line
1.8.3.1
author reviewer
#4 lore raw

PATCH 4/9 perf tools: Allow overriding sysfs and proc finding with env var

In reply to Jiri Olsa

From: Cody P Schafer <[email protected]>

SYSFS_PATH and PROC_PATH environment variables now let the user override the detection of sysfs and proc locations for testing purposes.

---
 tools/lib/api/fs/fs.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)
tools/lib/api/fs/fs.c +42 -1
index 5b5eb78..c1b49c3 100644
@@ -1,8 +1,10 @@
 /* TODO merge/factor in debugfs.c here */
 
+#include <ctype.h>
 #include <errno.h>
 #include <stdbool.h>
 #include <stdio.h>
+#include <stdlib.h>
 #include <string.h>
 #include <sys/vfs.h>
 
@@ -96,12 +98,51 @@ static bool fs__check_mounts(struct fs *fs)
 	return false;
 }
 
+static void mem_toupper(char *f, size_t len)
+{
+	while (len) {
+		*f = toupper(*f);
+		f++;
+		len--;
+	}
+}
+
+/*
+ * Check for "NAME_PATH" environment variable to override fs location (for
+ * testing). This matches the recommendation in Documentation/sysfs-rules.txt
+ * for SYSFS_PATH.
+ */
+static bool fs__env_override(struct fs *fs)
+{
+	char *override_path;
+	size_t name_len = strlen(fs->name);
+	/* name + "_PATH" + '\0' */
+	char upper_name[name_len + 5 + 1];
+	memcpy(upper_name, fs->name, name_len);
+	mem_toupper(upper_name, name_len);
+	strcpy(&upper_name[name_len], "_PATH");
+
+	override_path = getenv(upper_name);
+	if (!override_path)
+		return false;
+
+	fs->found = true;
+	strncpy(fs->path, override_path, sizeof(fs->path));
+	return true;
+}
+
 static const char *fs__get_mountpoint(struct fs *fs)
 {
+	if (fs__env_override(fs))
+		return fs->path;
+
 	if (fs__check_mounts(fs))
 		return fs->path;
 
-	return fs__read_mounts(fs) ? fs->path : NULL;
+	if (fs__read_mounts(fs))
+		return fs->path;
+
+	return NULL;
 }
 
 static const char *fs__mountpoint(int idx)
Signature · 1 line
1.8.3.1
author reviewer
#5 lore raw

PATCH 5/9 perf tools: Fix "==" into "=" in ui_browser__warning assignment

In reply to Jiri Olsa

From: zhangdianfang <[email protected]>

Convert "==" into "=" in ui_browser__warning assignment.

Bug description: https://bugzilla.kernel.org/show_bug.cgi?id=76751

---
 tools/perf/ui/browser.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/ui/browser.c +1 -1
index d11541d..3ccf6e1 100644
@@ -194,7 +194,7 @@ int ui_browser__warning(struct ui_browser *browser, int timeout,
 		ui_helpline__vpush(format, args);
 		va_end(args);
 	} else {
-		while ((key == ui__question_window("Warning!", text,
+		while ((key = ui__question_window("Warning!", text,
 						   "Press any key...",
 						   timeout)) == K_RESIZE)
 			ui_browser__handle_resize(browser);
Signature · 1 line
1.8.3.1
author reviewer
#6 lore raw

PATCH 6/9 perf tools: Remove elide setup for SORT_MODE__MEMORY mode

In reply to Jiri Olsa

There's no need to setup elide of sort_dso sort entry again with symbol_conf.dso_list list.

The only difference were list names of memory mode data, which does not make much sense to me.

Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Corey Ashford <[email protected]>
Cc David Ahern <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 tools/perf/util/sort.c | 13 -------------
 1 file changed, 13 deletions(-)
tools/perf/util/sort.c +0 -13
index 254f583..2aba620 100644
@@ -1412,19 +1412,6 @@ void sort__setup_elide(FILE *output)
 		sort_entry__setup_elide(&sort_sym_to,
 					symbol_conf.sym_to_list,
 					"sym_to", output);
-	} else if (sort__mode == SORT_MODE__MEMORY) {
-		sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list,
-					"symbol_daddr", output);
-		sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list,
-					"dso_daddr", output);
-		sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list,
-					"mem", output);
-		sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list,
-					"local_weight", output);
-		sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list,
-					"tlb", output);
-		sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list,
-					"snoop", output);
 	}
 
 	/*
Signature · 1 line
1.8.3.1
author reviewer
#7 lore raw

PATCH 7/9 perf tools: Move elide bool into perf_hpp_fmt struct

In reply to Jiri Olsa

After output/sort fields refactoring, it's expensive to check the elide bool in its current location inside the 'struct sort_entry'.

The perf_hpp__should_skip function gets highly noticable in workloads with high number of output/sort fields, like for:

  $ perf report -i perf-test.data -F overhead,sample,period,comm,pid,dso,symbol,cpu --stdio

Performance report:
9.70% perf [.] perf_hpp__should_skip

Moving the elide bool into the 'struct perf_hpp_fmt', which makes the perf_hpp__should_skip just single struct read.

Got speedup of around 22% for my test perf.data workload. The change should not harm any other workload types.

Performance counter stats for (10 runs):
before:

   358,319,732,626      cycles                    ( +-  0.55% )
   467,129,581,515      instructions              #    1.30  insns per cycle          ( +-  0.00% )
     150.943975206 seconds time elapsed           ( +-  0.62% )

now:

   278,785,972,990      cycles                    ( +-  0.12% )
   370,146,797,640      instructions              #    1.33  insns per cycle          ( +-  0.00% )
     116.416670507 seconds time elapsed           ( +-  0.31% )
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Corey Ashford <[email protected]>
Cc David Ahern <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 tools/perf/ui/browsers/hists.c |  8 ++--
 tools/perf/util/hist.h         |  8 +++-
 tools/perf/util/sort.c         | 90 ++++++++++++++++++++++++++----------------
 tools/perf/util/sort.h         |  2 +-
 4 files changed, 68 insertions(+), 40 deletions(-)
tools/perf/ui/browsers/hists.c +4 -4
index 5905acd..52c03fb 100644
@@ -1706,14 +1706,14 @@ zoom_dso:
 zoom_out_dso:
 				ui_helpline__pop();
 				browser->hists->dso_filter = NULL;
-				sort_dso.elide = false;
+				perf_hpp__set_elide(HISTC_DSO, false);
 			} else {
 				if (dso == NULL)
 					continue;
 				ui_helpline__fpush("To zoom out press <- or -> + \"Zoom out of %s DSO\"",
 						   dso->kernel ? "the Kernel" : dso->short_name);
 				browser->hists->dso_filter = dso;
-				sort_dso.elide = true;
+				perf_hpp__set_elide(HISTC_DSO, true);
 				pstack__push(fstack, &browser->hists->dso_filter);
 			}
 			hists__filter_by_dso(hists);
@@ -1725,13 +1725,13 @@ zoom_thread:
 zoom_out_thread:
 				ui_helpline__pop();
 				browser->hists->thread_filter = NULL;
-				sort_thread.elide = false;
+				perf_hpp__set_elide(HISTC_THREAD, false);
 			} else {
 				ui_helpline__fpush("To zoom out press <- or -> + \"Zoom out of %s(%d) thread\"",
 						   thread->comm_set ? thread__comm_str(thread) : "",
 						   thread->tid);
 				browser->hists->thread_filter = thread;
-				sort_thread.elide = true;
+				perf_hpp__set_elide(HISTC_THREAD, false);
 				pstack__push(fstack, &browser->hists->thread_filter);
 			}
 			hists__filter_by_thread(hists);
tools/perf/util/hist.h +7 -1
index 82b28ff..d2bf035 100644
@@ -205,6 +205,7 @@ struct perf_hpp_fmt {
 
 	struct list_head list;
 	struct list_head sort_list;
+	bool elide;
 };
 
 extern struct list_head perf_hpp__list;
@@ -252,7 +253,12 @@ void perf_hpp__append_sort_keys(void);
 
 bool perf_hpp__is_sort_entry(struct perf_hpp_fmt *format);
 bool perf_hpp__same_sort_entry(struct perf_hpp_fmt *a, struct perf_hpp_fmt *b);
-bool perf_hpp__should_skip(struct perf_hpp_fmt *format);
+
+static inline bool perf_hpp__should_skip(struct perf_hpp_fmt *format)
+{
+	return format->elide;
+}
+
 void perf_hpp__reset_width(struct perf_hpp_fmt *fmt, struct hists *hists);
 
 typedef u64 (*hpp_field_fn)(struct hist_entry *he);
tools/perf/util/sort.c +56 -34

131 lines not on this page · raw message

tools/perf/util/sort.h +1 -1
index 426b873..5bf0098 100644
@@ -202,7 +202,6 @@ struct sort_entry {
 	int	(*se_snprintf)(struct hist_entry *he, char *bf, size_t size,
 			       unsigned int width);
 	u8	se_width_idx;
-	bool	elide;
 };
 
 extern struct sort_entry sort_thread;
@@ -213,6 +212,7 @@ int setup_output_field(void);
 void reset_output_field(void);
 extern int sort_dimension__add(const char *);
 void sort__setup_elide(FILE *fp);
+void perf_hpp__set_elide(int idx, bool elide);
 
 int report_parse_ignore_callees_opt(const struct option *opt, const char *arg, int unset);
 
Signature · 1 line
1.8.3.1
author reviewer
#8 lore raw

PATCH 8/9 perf record: Fix poll return value propagation

In reply to Jiri Olsa

If the perf record command is interrupted in record__mmap_read_all function, the 'done' is set and err has the latest poll return value, which is most likely positive number (= number of pollfds ready to read).

This 'positive err' is then propagated to the exit code, resulting in not finishing the perf.data header properly, causing following error in report:

  # perf record -F 50000 -a

---
make the system real busy, so there's more chance to interrupt perf in event writing code
---

^C[ perf record: Woken up 16 times to write data ]
[ perf record: Captured and wrote 30.292 MB perf.data (~1323468 samples) ]

  # perf report --stdio > /dev/null
  WARNING: The perf.data file's data size field is 0 which is unexpected.
  Was the 'perf record' command properly terminated?

Fixing this by checking for positive poll return value and setting err to 0.

Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Corey Ashford <[email protected]>
Cc David Ahern <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 tools/perf/builtin-record.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/perf/builtin-record.c +5 -1
index e4c85b8..ce2cfec 100644
@@ -454,7 +454,11 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 			if (done)
 				break;
 			err = poll(rec->evlist->pollfd, rec->evlist->nr_fds, -1);
-			if (err < 0 && errno == EINTR)
+			/*
+			 * Propagate error, only if there's any. Ignore positive
+			 * number of returned events and interrupt error.
+			 */
+			if (err > 0 || (err < 0 && errno == EINTR))
 				err = 0;
 			waking++;
 		}
Signature · 1 line
1.8.3.1
author reviewer
#9 lore raw

PATCH 9/9 perf tools: Fix 'make help' message error

In reply to Jiri Olsa

From: Jianyu Zhan <[email protected]>

Currently 'make help' message has such hint:

use "make prefix=<path> <install target>" to install to a particular
path like make prefix=/usr/local install install-doc

But this is misleading, when I specify "prefix=/usr/local", it has got no respect at all.

This is because that, "DESTDIR" is considered first. In this case, "DESTDIR" has an empty value, so "prefix" is honored. However, "prefix" is unconditionally assigned to $HOME, regardless of what it is set to from command line. So our "prefix" setting got no respect and the actual destination falls back to $HOME.

This patch fixes this issue and corrects the help message.

---
 tools/perf/Makefile.perf   | 4 ++--
 tools/perf/config/Makefile | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
tools/perf/Makefile.perf +2 -2
index 6286e13..ae20edf 100644
@@ -789,8 +789,8 @@ help:
 	@echo ''
 	@echo 'Perf install targets:'
 	@echo '  NOTE: documentation build requires asciidoc, xmlto packages to be installed'
-	@echo '  HINT: use "make prefix=<path> <install target>" to install to a particular'
-	@echo '        path like make prefix=/usr/local install install-doc'
+	@echo '  HINT: use "prefix" or "DESTDIR" to install to a particular'
+	@echo '        path like "make prefix=/usr/local install install-doc"'
 	@echo '  install	- install compiled binaries'
 	@echo '  install-doc	- install *all* documentation'
 	@echo '  install-man	- install manpage documentation'
tools/perf/config/Makefile +1 -1
index 319426f..4f100b5 100644
@@ -600,7 +600,7 @@ endif
 
 # Make the path relative to DESTDIR, not to prefix
 ifndef DESTDIR
-prefix = $(HOME)
+prefix ?= $(HOME)
 endif
 bindir_relative = bin
 bindir = $(prefix)/$(bindir_relative)
Signature · 1 line
1.8.3.1
#10 lore raw
In reply to Jiri Olsa
Quoted 63 lines · Jiri Olsa

hi Ingo,
please consider pulling

thanks,
jirka

The following changes since commit 9b261365dd73a5014b49033327ad881708e81f33:

Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf into perf/core (2014-06-03 20:22:40 +0200)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git tags/perf-core-for-mingo

for you to fetch changes up to fc9cabeabf42d76854059e7bce81a02645e7e5ca:

perf tools: Fix 'make help' message error (2014-06-03 21:35:12 +0200)

----------------------------------------------------------------
perf/core improvements and fixes:

. Warn the user when trace command is not available (Arnaldo Carvalho de Melo)

. Add warning when disabling perl scripting support due to missing devel files (Arnaldo Carvalho de Melo)

. Consider header files outside perf directory in tags target (Sebastian Andrzej Siewior)

. Allow overriding sysfs and proc finding with env var (Cody P Schafer)

. Fix "==" into "=" in ui_browser__warning assignment (zhangdianfang)

. Factor elide bool handling in sort code (Jiri Olsa)

. Fix poll return value propagation (Jiri Olsa)

. Fix 'make help' message error (Jianyu Zhan)

----------------------------------------------------------------
Arnaldo Carvalho de Melo (2):
perf trace: Warn the user when not available
perf tools: Add warning when disabling perl scripting support due to missing devel files

Cody P Schafer (1):
perf tools: Allow overriding sysfs and proc finding with env var

Jianyu Zhan (1):
perf tools: Fix 'make help' message error

Jiri Olsa (3):
perf tools: Remove elide setup for SORT_MODE__MEMORY mode perf tools: Move elide bool into perf_hpp_fmt struct perf record: Fix poll return value propagation

Sebastian Andrzej Siewior (1):
perf tools: Consider header files outside perf directory in tags target

zhangdianfang (1):

      perf tools: Fix "==" into "=" in ui_browser__warning assignment
Quoted 11 lines
 tools/lib/api/fs/fs.c          |  43 ++++++++++++++++-
 tools/perf/Makefile.perf       |  13 ++++--
 tools/perf/builtin-record.c    |   6 ++-
 tools/perf/config/Makefile     |   3 +-
 tools/perf/perf.c              |   8 +++-
 tools/perf/ui/browser.c        |   2 +-
 tools/perf/ui/browsers/hists.c |   8 ++--
 tools/perf/util/hist.h         |   8 +++-
 tools/perf/util/sort.c         | 103 ++++++++++++++++++++++-------------------
 tools/perf/util/sort.h         |   2 +-
 10 files changed, 132 insertions(+), 64 deletions(-)

Pulled, thanks a lot Jiri!

Ingo