perf/core improvements and fixes

Hi Ingo,

Please consider pulling,

- Arnaldo

The following changes since commit c88f2096136416b261bd3647cc260935f6e95805:

perf: Do not check PERF_EVENT_STATE_EXIT on syscall read path (2014-09-16 10:30:36 +0200)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo

for you to fetch changes up to e5685730e2c620f97bc12380e9370e857e5bd7a7:

perf record: Use ring buffer consume method to look like other tools (2014-09-17 18:01:43 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

User visible:

o Add +field argument support for --sort option (Jiri Olsa)

o Do not access kallsyms when analyzing user binaries with 'probe' (Masami Hiramatsu)

o Ignore stripped vmlinux and fallback to kallsyms (Anton Blanchard)

o Add path to Ubuntu kernel debuginfo file (Anton Blanchard)

o Disable kernel symbol demangling by default (Avi Kivity)

Infrastructure:

o More intel PT prep work, from Adrian Hunter, including:

- Let a user specify a PMU event without any config terms
- Add perf-with-kcore script
- Let default config be defined for a PMU
- Add perf_pmu__scan_file()

o "perf kvm stat report" improvements by Alexander Yarygin:

  o  Save pid string in opts.target.pid
  o  Enable the target.system_wide flag
  o  Unify the title bar output

o Fix build issue on powerpc when DWARF support is disabled (Anton Blanchard)

o Allow to specify lib compile variable for spec usage (Jiri Olsa)

o Fix build on ARM (Stephane Eranian)

o Fix build on powerpc when DWARF support is disabled (Anton Blanchard)

o Don't include sys/poll.h directly (Arnaldo Carvalho de Melo)

o Use ring buffer consume method to look like other tools (Arnaldo Carvalho de Melo)

Chanho Park (1):
perf tools: define _DEFAULT_SOURCE for glibc_2.20

o Allow to specify lib compile variable for spec usage (Jiri Olsa)

o Fix GNU-only grep usage in Makefile (John Spencer)

----------------------------------------------------------------
Adrian Hunter (4):
perf tools: Let a user specify a PMU event without any config terms perf tools: Add perf-with-kcore script
perf tools: Let default config be defined for a PMU perf tools: Add perf_pmu__scan_file()

Alexander Yarygin (3):
perf kvm stat report: Save pid string in opts.target.pid perf kvm stat report: Enable the target.system_wide flag perf kvm stat report: Unify the title bar output

Anton Blanchard (3):
perf tools powerpc: Fix build issue when DWARF support is disabled perf symbols: Ignore stripped vmlinux and fallback to kallsyms perf symbols: Add path to Ubuntu kernel debuginfo file

Arnaldo Carvalho de Melo (2):
perf tools: Don't include sys/poll.h directly
perf record: Use ring buffer consume method to look like other tools

Avi Kivity (1):
perf tools: Disable kernel symbol demangling by default

Chanho Park (1):
perf tools: define _DEFAULT_SOURCE for glibc_2.20

Jiri Olsa (2):
perf tools: Add +field argument support for --sort option perf tools: Allow to specify lib compile variable for spec usage

John Spencer (1):
perf tools: Fix GNU-only grep usage in Makefile

Masami Hiramatsu (2):
perf probe: Do not access kallsyms when analyzing user binaries perf probe: Do not use dwfl_module_addrsym if dwarf_diename finds symbol name

Stephane Eranian (1):
perf tool: fix compilation for ARM

 tools/perf/.gitignore                       |   1 +
 tools/perf/Documentation/perf-probe.txt     |   3 +
 tools/perf/Documentation/perf-report.txt    |   3 +
 tools/perf/Documentation/perf-top.txt       |   3 +
 tools/perf/Makefile.perf                    |   5 +-
 tools/perf/arch/arm/tests/dwarf-unwind.c    |   1 +
 tools/perf/arch/arm/util/unwind-libunwind.c |   1 +
 tools/perf/arch/powerpc/Makefile            |   2 +-
 tools/perf/bench/sched-messaging.c          |   2 +-
 tools/perf/builtin-kvm.c                    |  23 +--
 tools/perf/builtin-probe.c                  |   5 +-
 tools/perf/builtin-record.c                 |   8 +-
 tools/perf/builtin-report.c                 |   2 +
 tools/perf/builtin-top.c                    |   4 +-
 tools/perf/config/Makefile                  |  12 +-
 tools/perf/config/utilities.mak             |   2 +-
 tools/perf/perf-with-kcore.sh               | 259 ++++++++++++++++++++++++++++
 tools/perf/tests/pmu.c                      |   2 +-
 tools/perf/util/kvm-stat.h                  |   1 -
 tools/perf/util/parse-events.c              |  13 +-
 tools/perf/util/parse-events.y              |  10 ++
 tools/perf/util/pmu.c                       |  79 +++++++--
 tools/perf/util/pmu.h                       |  12 +-
 tools/perf/util/probe-event.c               |   9 +-
 tools/perf/util/probe-event.h               |   3 +-
 tools/perf/util/probe-finder.c              |  16 +-
 tools/perf/util/sort.c                      |  37 +++-
 tools/perf/util/symbol-elf.c                |  15 +-
 tools/perf/util/symbol.c                    |   9 +-
 tools/perf/util/symbol.h                    |   1 +
 tools/perf/util/util.h                      |   4 +-
 31 files changed, 487 insertions(+), 60 deletions(-)
 create mode 100644 tools/perf/perf-with-kcore.sh

From: Jiri Olsa <[email protected]>

Adding support to add field(s) to default sort order via using the '+' prefix, like for report:

  $ perf report
  Samples: 2K of event 'cycles', Event count (approx.): 882172583
  Overhead  Command  Shared Object        Symbol
     7.39%  swapper  [kernel.kallsyms]    [k] intel_idle
     1.97%  firefox  libpthread-2.17.so   [.] pthread_mutex_lock
     1.39%  firefox  [snd_hda_intel]      [k] azx_get_position
     1.11%  firefox  libpthread-2.17.so   [.] pthread_mutex_unlock

  $ perf report -s +cpu
  Samples: 2K of event 'cycles', Event count (approx.): 882172583
  Overhead  Command  Shared Object        Symbol                  CPU
     2.89%  swapper  [kernel.kallsyms]    [k] intel_idle          000
     2.61%  swapper  [kernel.kallsyms]    [k] intel_idle          002
     1.20%  swapper  [kernel.kallsyms]    [k] intel_idle          001
     0.82%  firefox  libpthread-2.17.so   [.] pthread_mutex_lock  002

Works in general for commands using --sort option.

v2 with changes suggested:
- Use dynamic memory instead static buffer
- Fix error message typo

---
 tools/perf/util/sort.c | 37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)

index 1958637cf136..289df9d1e65a 100644
@@ -1446,12 +1446,47 @@ static const char *get_default_sort_order(void)
 	return default_sort_orders[sort__mode];
 }
 
+static int setup_sort_order(void)
+{
+	char *new_sort_order;
+
+	/*
+	 * Append '+'-prefixed sort order to the default sort
+	 * order string.
+	 */
+	if (!sort_order || is_strict_order(sort_order))
+		return 0;
+
+	if (sort_order[1] == '\0') {
+		error("Invalid --sort key: `+'");
+		return -EINVAL;
+	}
+
+	/*
+	 * We allocate new sort_order string, but we never free it,
+	 * because it's checked over the rest of the code.
+	 */
+	if (asprintf(&new_sort_order, "%s,%s",
+		     get_default_sort_order(), sort_order + 1) < 0) {
+		error("Not enough memory to set up --sort");
+		return -ENOMEM;
+	}
+
+	sort_order = new_sort_order;
+	return 0;
+}
+
 static int __setup_sorting(void)
 {
 	char *tmp, *tok, *str;
-	const char *sort_keys = sort_order;
+	const char *sort_keys;
 	int ret = 0;
 
+	ret = setup_sort_order();
+	if (ret)
+		return ret;
+
+	sort_keys = sort_order;
 	if (sort_keys == NULL) {
 		if (is_strict_order(field_order)) {
 			/*

1.9.3

From: Anton Blanchard <[email protected]>

The powerpc skip callchain code uses DWARF, so we must disable it if DWARF is disabled.

---
 tools/perf/arch/powerpc/Makefile |  2 +-
 tools/perf/config/Makefile       | 10 ++++++----
 2 files changed, 7 insertions(+), 5 deletions(-)

index b92219b1900d..6f7782bea5dd 100644
@@ -1,6 +1,6 @@
 ifndef NO_DWARF
 PERF_HAVE_DWARF_REGS := 1
 LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/dwarf-regs.o
+LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/skip-callchain-idx.o
 endif
 LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/header.o
-LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/skip-callchain-idx.o

index 75d4c237b03d..98c9fd174e98 100644
@@ -48,10 +48,6 @@ ifneq ($(ARCH),$(filter $(ARCH),x86 arm))
   NO_LIBDW_DWARF_UNWIND := 1
 endif
 
-ifeq ($(ARCH),powerpc)
-  CFLAGS += -DHAVE_SKIP_CALLCHAIN_IDX
-endif
-
 ifeq ($(LIBUNWIND_LIBS),)
   NO_LIBUNWIND := 1
 else

@@ -378,6 +374,12 @@ ifndef NO_LIBELF
   endif # NO_DWARF
 endif # NO_LIBELF
 
+ifeq ($(ARCH),powerpc)
+  ifndef NO_DWARF
+    CFLAGS += -DHAVE_SKIP_CALLCHAIN_IDX
+  endif
+endif
+
 ifndef NO_LIBUNWIND
   ifneq ($(feature-libunwind), 1)
     msg := $(warning No libunwind found. Please install libunwind-dev[el] >= 1.1 and/or set LIBUNWIND_DIR);

1.9.3

From: Alexander Yarygin <[email protected]>

The 'perf kvm stat report' command uses the kvm->pid_str field to keep the value of the --pid option. Let's use kvm->opts.target.pid instead.

---
 tools/perf/builtin-kvm.c   | 6 +++---
 tools/perf/util/kvm-stat.h | 1 -
 2 files changed, 3 insertions(+), 4 deletions(-)

index 1a4ef9cd9d5f..646ec5d5de80 100644
@@ -1085,8 +1085,8 @@ static int read_events(struct perf_kvm_stat *kvm)
 
 static int parse_target_str(struct perf_kvm_stat *kvm)
 {
-	if (kvm->pid_str) {
-		kvm->pid_list = intlist__new(kvm->pid_str);
+	if (kvm->opts.target.pid) {
+		kvm->pid_list = intlist__new(kvm->opts.target.pid);
 		if (kvm->pid_list == NULL) {
 			pr_err("Error parsing process id string\n");
 			return -EINVAL;

@@ -1188,7 +1188,7 @@ kvm_events_report(struct perf_kvm_stat *kvm, int argc, const char **argv)
 		OPT_STRING('k', "key", &kvm->sort_key, "sort-key",
 			    "key for sorting: sample(sort by samples number)"
 			    " time (sort by avg time)"),
-		OPT_STRING('p', "pid", &kvm->pid_str, "pid",
+		OPT_STRING('p', "pid", &kvm->opts.target.pid, "pid",
 			   "analyze events only for given process id(s)"),
 		OPT_END()
 	};

index 0b5a8cd2ee79..cf1d7913783b 100644
@@ -92,7 +92,6 @@ struct perf_kvm_stat {
 	u64 lost_events;
 	u64 duration;
 
-	const char *pid_str;
 	struct intlist *pid_list;
 
 	struct rb_root result;

1.9.3

From: Alexander Yarygin <[email protected]>

The 'perf kvm stat report' command can be used to analyze events either for system wide or for specific pids.

Let's enable kvm->opts.target.system_wide flag when 'report' command is running for system-wide analyzing. This helps to sync kvm->opts.target values in 'report' and 'live' commands.

---
 tools/perf/builtin-kvm.c | 3 +++
 1 file changed, 3 insertions(+)

index 646ec5d5de80..84295ab6c4e1 100644
@@ -1207,6 +1207,9 @@ kvm_events_report(struct perf_kvm_stat *kvm, int argc, const char **argv)
 					   kvm_events_report_options);
 	}
 
+	if (!kvm->opts.target.pid)
+		kvm->opts.target.system_wide = true;
+
 	return kvm_events_report_vcpu(kvm);
 }
 

1.9.3

From: Alexander Yarygin <[email protected]>

The 'live' command prints additional information to the "Analyze events for " title bar about the current target. Let's print the same title for the 'report' command.

---
 tools/perf/builtin-kvm.c | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

index 84295ab6c4e1..f5d3ae483110 100644
@@ -543,14 +543,12 @@ static void print_vcpu_info(struct perf_kvm_stat *kvm)
 
 	pr_info("Analyze events for ");
 
-	if (kvm->live) {
-		if (kvm->opts.target.system_wide)
-			pr_info("all VMs, ");
-		else if (kvm->opts.target.pid)
-			pr_info("pid(s) %s, ", kvm->opts.target.pid);
-		else
-			pr_info("dazed and confused on what is monitored, ");
-	}
+	if (kvm->opts.target.system_wide)
+		pr_info("all VMs, ");
+	else if (kvm->opts.target.pid)
+		pr_info("pid(s) %s, ", kvm->opts.target.pid);
+	else
+		pr_info("dazed and confused on what is monitored, ");
 
 	if (vcpu == -1)
 		pr_info("all VCPUs:\n\n");

1.9.3

From: Jiri Olsa <[email protected]>

We need a way to specify $(lib) part of the installation path for traceevent plugin libraries. Currently we use 'lib64' for x86_64 and 'lib' otherwise.

Instead of listing all possible values, this change allows the rpm spec code to specify the correct $(lib) part based on processed architecture, like

  $ make ... lib=%{_lib}

---
 tools/perf/config/Makefile | 2 ++
 1 file changed, 2 insertions(+)

index 98c9fd174e98..58f609198c6d 100644
@@ -653,11 +653,13 @@ else
 sysconfdir = $(prefix)/etc
 ETC_PERFCONFIG = etc/perfconfig
 endif
+ifndef lib
 ifeq ($(IS_X86_64),1)
 lib = lib64
 else
 lib = lib
 endif
+endif # lib
 libdir = $(prefix)/$(lib)
 
 # Shell quote (do not use $(call) to accommodate ancient setups);

1.9.3

From: Adrian Hunter <[email protected]>

This enables a PMU event to be specified in the form:

	pmu//

which is effectively the same as:

	pmu/config=0/

This patch is a precursor to defining default config for a PMU.

Further explanation extracted from lkml thread:

Imagine that the 'tsc' term did not exist.

Intel PT trace data would not contain TSC packets, and the decoder would not know how to decode them.

Then imagine that a new version of the hardware adds 'tsc'.

It is such a useful feature that we want it by default, but older versions of the tools don't know how to decode it, so the kernel cannot turn it on by default.

It is similar to why the kernel does not select perf_event_attr.mmap2 by default.

The kernel doesn't know whether the tool supports it.

---
 tools/perf/util/parse-events.c |  6 ++++++
 tools/perf/util/parse-events.y | 10 ++++++++++
 2 files changed, 16 insertions(+)

index e34c81a0bcf3..e75628813968 100644
@@ -645,6 +645,12 @@ int parse_events_add_pmu(struct list_head *list, int *idx,
 
 	memset(&attr, 0, sizeof(attr));
 
+	if (!head_config) {
+		attr.type = pmu->type;
+		evsel = __add_event(list, idx, &attr, NULL, pmu->cpus);
+		return evsel ? 0 : -ENOMEM;
+	}
+
 	if (perf_pmu__check_alias(pmu, head_config, &unit, &scale))
 		return -EINVAL;
 

index 0bc87ba46bf3..55fab6ad609a 100644
@@ -210,6 +210,16 @@ PE_NAME '/' event_config '/'
 	parse_events__free_terms($3);
 	$$ = list;
 }
+|
+PE_NAME '/' '/'
+{
+	struct parse_events_evlist *data = _data;
+	struct list_head *list;
+
+	ALLOC_LIST(list);
+	ABORT_ON(parse_events_add_pmu(list, &data->idx, $1, NULL));
+	$$ = list;
+}
 
 value_sym:
 PE_VALUE_SYM_HW

1.9.3

From: Adrian Hunter <[email protected]>

Decoding an Intel PT trace of the kernel requires an accurate kernel object image. This is provided by making a copy of kcore. However the copy needs to be made under the same conditions as the original recording, and then it needs to be associated with the perf.data file. The perf-with-kcore script does that.

The script also checks the permissions on the buildid cache and can be used to fix them. That is needed for distributions where root does not have a home directory and consequently writes to the same buildid cache as the user, resulting in cached files that the user does not have access to.

Example:

$ ./perf-with-kcore
Usage: perf-with-kcore <perf sub-command> <perf.data directory> [<sub-command options> [ -- <workload>]]
       <perf sub-command> can be record, script, report or inject
   or: perf-with-kcore fix_buildid_cache_permissions
$ ./perf-with-kcore record pt_uname -e intel_pt// -- uname
Recording
Using /home/ahunter/bin/perf
perf version 3.15.rc3.g4549ba
/home/ahunter/bin/perf record -o pt_uname/perf.data -e intel_pt//  -- uname
Linux
[ perf record: Woken up 3 times to write data ]
[ perf record: Captured and wrote 0.023 MB pt_uname/perf.data ]
Copying kcore
[sudo] password for ahunter:
Done
$ tools/perf/perf-with-kcore.sh script pt_uname | head
Using /home/ahunter/bin/perf
perf version 3.15.rc3.g4549ba
/home/ahunter/bin/perf script -i pt_uname/perf.data --kallsyms=pt_uname/kcore_dir/kallsyms
         swapper     0 [002] 161533.969666: sched:sched_switch: swapper/2:0 [120] R ==> perf:11316 [120]
          :11315 11315 [003] 161533.969704: sched:sched_switch: perf:11315 [120] S ==> swapper/3:0 [120]
          :11316 11316 [002] 161533.969783: sched:sched_switch: perf:11316 [120] R ==> migration/2:33 [0]
             :33    33 [002] 161533.969791: sched:sched_switch: migration/2:33 [0] S ==> swapper/2:0 [120]
         swapper     0 [003] 161533.969792: sched:sched_switch: swapper/3:0 [120] R ==> perf:11316 [120]
          :11316 11316 [003] 161533.970062: branches:                 0 [unknown] ([unknown]) => ffffffff810532fa native_write_msr_safe ([kernel.kallsyms])
          :11316 11316 [003] 161533.970062: branches:  ffffffff810532fd native_write_msr_safe ([kernel.kallsyms]) => ffffffff81035b31 pt_config_start ([kernel.kallsyms])

---
 tools/perf/.gitignore         |   1 +
 tools/perf/Makefile.perf      |   5 +-
 tools/perf/perf-with-kcore.sh | 259 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 264 insertions(+), 1 deletion(-)
 create mode 100644 tools/perf/perf-with-kcore.sh

index 782d86e961b9..717221e98450 100644
@@ -15,6 +15,7 @@ perf.data
 perf.data.old
 output.svg
 perf-archive
+perf-with-kcore
 tags
 TAGS
 cscope*

index 95e832b1bc3c..171f4e65601b 100644
@@ -126,6 +126,7 @@ PYRF_OBJS =
 SCRIPT_SH =
 
 SCRIPT_SH += perf-archive.sh
+SCRIPT_SH += perf-with-kcore.sh
 
 grep-libs = $(filter -l%,$(1))
 strip-libs = $(filter-out -l%,$(1))

@@ -878,6 +879,8 @@ install-bin: all install-gtk
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)'
 	$(call QUIET_INSTALL, perf-archive) \
 		$(INSTALL) $(OUTPUT)perf-archive -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)'
+	$(call QUIET_INSTALL, perf-with-kcore) \
+		$(INSTALL) $(OUTPUT)perf-with-kcore -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)'
 ifndef NO_LIBPERL
 	$(call QUIET_INSTALL, perl-scripts) \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl/Perf-Trace-Util/lib/Perf/Trace'; \

@@ -923,7 +926,7 @@ config-clean:
 	@$(MAKE) -C config/feature-checks clean >/dev/null
 
 clean: $(LIBTRACEEVENT)-clean $(LIBAPIKFS)-clean config-clean
-	$(call QUIET_CLEAN, core-objs)  $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf.o $(LANG_BINDINGS) $(GTK_OBJS)
+	$(call QUIET_CLEAN, core-objs)  $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf-with-kcore $(OUTPUT)perf.o $(LANG_BINDINGS) $(GTK_OBJS)
 	$(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf
 	$(call QUIET_CLEAN, core-gen)   $(RM)  *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope* $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)PERF-CFLAGS $(OUTPUT)PERF-FEATURES $(OUTPUT)util/*-bison* $(OUTPUT)util/*-flex*
 	$(QUIET_SUBDIR0)Documentation $(QUIET_SUBDIR1) clean

1.9.3

From: Adrian Hunter <[email protected]>

This allows default config terms to be provided for a PMU. So, for example, when the Intel PT PMU is added, it will be possible to specify:

	intel_pt//

which will be the same as:

	intel_pt/tsc=1,noretcomp=0/

meaning that the trace should contain TSC timestamps and perform 'return compression'.

An important consideration of this patch is that it must be possible to overwrite the default values. That has meant changing the logic so that a zero value can replace a non-zero value.

---
 tools/perf/tests/pmu.c         |  2 +-
 tools/perf/util/parse-events.c |  7 ++++++-
 tools/perf/util/pmu.c          | 42 ++++++++++++++++++++++++++----------------
 tools/perf/util/pmu.h          |  9 ++++++++-
 4 files changed, 41 insertions(+), 19 deletions(-)

index 12b322fa3475..eeb68bb1972d 100644
@@ -152,7 +152,7 @@ int test__pmu(void)
 		if (ret)
 			break;
 
-		ret = perf_pmu__config_terms(&formats, &attr, terms);
+		ret = perf_pmu__config_terms(&formats, &attr, terms, false);
 		if (ret)
 			break;
 

index e75628813968..61be3e695ec2 100644
@@ -643,7 +643,12 @@ int parse_events_add_pmu(struct list_head *list, int *idx,
 	if (!pmu)
 		return -EINVAL;
 
-	memset(&attr, 0, sizeof(attr));
+	if (pmu->default_config) {
+		memcpy(&attr, pmu->default_config,
+		       sizeof(struct perf_event_attr));
+	} else {
+		memset(&attr, 0, sizeof(attr));
+	}
 
 	if (!head_config) {
 		attr.type = pmu->type;

index 1c1e2eecbe1f..413b9a63c38d 100644
@@ -13,9 +13,12 @@ enum {
 
 #define PERF_PMU_FORMAT_BITS 64
 
+struct perf_event_attr;
+
 struct perf_pmu {
 	char *name;
 	__u32 type;
+	struct perf_event_attr *default_config;
 	struct cpu_map *cpus;
 	struct list_head format;  /* HEAD struct perf_pmu_format -> list */
 	struct list_head aliases; /* HEAD struct perf_pmu_alias -> list */

@@ -27,7 +30,8 @@ int perf_pmu__config(struct perf_pmu *pmu, struct perf_event_attr *attr,
 		     struct list_head *head_terms);
 int perf_pmu__config_terms(struct list_head *formats,
 			   struct perf_event_attr *attr,
-			   struct list_head *head_terms);
+			   struct list_head *head_terms,
+			   bool zero);
 int perf_pmu__check_alias(struct perf_pmu *pmu, struct list_head *head_terms,
 			  const char **unit, double *scale);
 struct list_head *perf_pmu__alias(struct perf_pmu *pmu,

@@ -46,4 +50,7 @@ void print_pmu_events(const char *event_glob, bool name_only);
 bool pmu_have_event(const char *pname, const char *name);
 
 int perf_pmu__test(void);
+
+struct perf_event_attr *perf_pmu__get_default_config(struct perf_pmu *pmu);
+
 #endif /* __PMU_H */

1.9.3

From: Adrian Hunter <[email protected]>

Add a function to scan a sysfs file within the pmu device directory.

This will be used to read capability values from the PMU 'caps' subdirectory.

---
 tools/perf/util/pmu.c | 37 +++++++++++++++++++++++++++++++++++++
 tools/perf/util/pmu.h |  3 +++
 2 files changed, 40 insertions(+)

index 438bb261f391..22a4ad5a927a 100644
@@ -3,6 +3,7 @@
 #include <unistd.h>
 #include <stdio.h>
 #include <stdbool.h>
+#include <stdarg.h>
 #include <dirent.h>
 #include <api/fs/fs.h>
 #include <locale.h>

@@ -804,3 +805,39 @@ bool pmu_have_event(const char *pname, const char *name)
 	}
 	return false;
 }
+
+static FILE *perf_pmu__open_file(struct perf_pmu *pmu, const char *name)
+{
+	struct stat st;
+	char path[PATH_MAX];
+	const char *sysfs;
+
+	sysfs = sysfs__mountpoint();
+	if (!sysfs)
+		return NULL;
+
+	snprintf(path, PATH_MAX,
+		 "%s" EVENT_SOURCE_DEVICE_PATH "%s/%s", sysfs, pmu->name, name);
+
+	if (stat(path, &st) < 0)
+		return NULL;
+
+	return fopen(path, "r");
+}
+
+int perf_pmu__scan_file(struct perf_pmu *pmu, const char *name, const char *fmt,
+			...)
+{
+	va_list args;
+	FILE *file;
+	int ret = EOF;
+
+	va_start(args, fmt);
+	file = perf_pmu__open_file(pmu, name);
+	if (file) {
+		ret = vfscanf(file, fmt, args);
+		fclose(file);
+	}
+	va_end(args);
+	return ret;
+}

index 413b9a63c38d..0f5c0a88fdc8 100644
@@ -49,6 +49,9 @@ struct perf_pmu *perf_pmu__scan(struct perf_pmu *pmu);
 void print_pmu_events(const char *event_glob, bool name_only);
 bool pmu_have_event(const char *pname, const char *name);
 
+int perf_pmu__scan_file(struct perf_pmu *pmu, const char *name, const char *fmt,
+			...) __attribute__((format(scanf, 3, 4)));
+
 int perf_pmu__test(void);
 
 struct perf_event_attr *perf_pmu__get_default_config(struct perf_pmu *pmu);

1.9.3

From: Stephane Eranian <[email protected]>

This patch fixes ARM compile of the perf tool. The debug.h header file was missing from a couple of unwind related modules.

---
 tools/perf/arch/arm/tests/dwarf-unwind.c    | 1 +
 tools/perf/arch/arm/util/unwind-libunwind.c | 1 +
 2 files changed, 2 insertions(+)

index 9f870d27cb39..62eff847f91c 100644
@@ -3,6 +3,7 @@
 #include "thread.h"
 #include "map.h"
 #include "event.h"
+#include "debug.h"
 #include "tests/tests.h"
 
 #define STACK_SIZE 8192

index 729ed69a6664..62c397ed3d97 100644
@@ -3,6 +3,7 @@
 #include <libunwind.h>
 #include "perf_regs.h"
 #include "../../util/unwind.h"
+#include "../../util/debug.h"
 
 int libunwind__arch_reg_id(int regnum)
 {

1.9.3

From: Avi Kivity <[email protected]>

Some Linux symbols (for example __vt_event_wait) are interpreted by the demangler as C++ mangled names, which of course they aren't.

Disable kernel symbol demangling by default to avoid this, and allow enabling it with a new option --demangle-kernel for those who wish it.

---
 tools/perf/Documentation/perf-probe.txt  | 3 +++
 tools/perf/Documentation/perf-report.txt | 3 +++
 tools/perf/Documentation/perf-top.txt    | 3 +++
 tools/perf/builtin-probe.c               | 2 ++
 tools/perf/builtin-report.c              | 2 ++
 tools/perf/builtin-top.c                 | 2 ++
 tools/perf/util/symbol-elf.c             | 7 ++++++-
 tools/perf/util/symbol.c                 | 1 +
 tools/perf/util/symbol.h                 | 1 +
 9 files changed, 23 insertions(+), 1 deletion(-)

index 1513935c399b..aaa869be3dc1 100644
@@ -104,6 +104,9 @@ OPTIONS
 	Specify path to the executable or shared library file for user
 	space tracing. Can also be used with --funcs option.
 
+--demangle-kernel::
+	Demangle kernel symbols.
+
 In absence of -m/-x options, perf probe checks if the first argument after
 the options is an absolute path name. If its an absolute path, perf probe
 uses it as a target module/target user space binary to probe.

index d561e0214f52..0927bf4e6c2a 100644
@@ -276,6 +276,9 @@ OPTIONS
 	Demangle symbol names to human readable form. It's enabled by default,
 	disable with --no-demangle.
 
+--demangle-kernel::
+	Demangle kernel symbol names to human readable form (for C++ kernels).
+
 --mem-mode::
 	Use the data addresses of samples in addition to instruction addresses
 	to build the histograms.  To generate meaningful output, the perf.data

index 28fdee394880..3265b1070518 100644
@@ -98,6 +98,9 @@ Default is to monitor all CPUS.
 --hide_user_symbols::
         Hide user symbols.
 
+--demangle-kernel::
+        Demangle kernel symbols.
+
 -D::
 --dump-symtab::
         Dump the symbol table used for profiling.

index 347729e29a92..4d6858dbebea 100644
@@ -376,6 +376,8 @@ __cmd_probe(int argc, const char **argv, const char *prefix __maybe_unused)
 			"target executable name or path", opt_set_target),
 	OPT_BOOLEAN(0, "demangle", &symbol_conf.demangle,
 		    "Disable symbol demangling"),
+	OPT_BOOLEAN(0, "demangle-kernel", &symbol_conf.demangle_kernel,
+		    "Enable kernel symbol demangling"),
 	OPT_END()
 	};
 	int ret;

index 3da59a87ec7c..8c0b3f22412a 100644
@@ -680,6 +680,8 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
 		   "objdump binary to use for disassembly and annotations"),
 	OPT_BOOLEAN(0, "demangle", &symbol_conf.demangle,
 		    "Disable symbol demangling"),
+	OPT_BOOLEAN(0, "demangle-kernel", &symbol_conf.demangle_kernel,
+		    "Enable kernel symbol demangling"),
 	OPT_BOOLEAN(0, "mem-mode", &report.mem_mode, "mem access profile"),
 	OPT_CALLBACK(0, "percent-limit", &report, "percent",
 		     "Don't show entries under that percent", parse_percent_limit),

index 9848e270b92c..7da2c46ea38f 100644
@@ -1142,6 +1142,8 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)
 		    "Interleave source code with assembly code (default)"),
 	OPT_BOOLEAN(0, "asm-raw", &symbol_conf.annotate_asm_raw,
 		    "Display raw encoding of assembly instructions (default)"),
+	OPT_BOOLEAN(0, "demangle-kernel", &symbol_conf.demangle_kernel,
+		    "Enable kernel symbol demangling"),
 	OPT_STRING(0, "objdump", &objdump_path, "path",
 		    "objdump binary to use for disassembly and annotations"),
 	OPT_STRING('M', "disassembler-style", &disassembler_style, "disassembler style",

index 9fb5e9e9f161..9c9b27fbc78d 100644
@@ -680,6 +680,11 @@ static u64 ref_reloc(struct kmap *kmap)
 	return 0;
 }
 
+static bool want_demangle(bool is_kernel_sym)
+{
+	return is_kernel_sym ? symbol_conf.demangle_kernel : symbol_conf.demangle;
+}
+
 int dso__load_sym(struct dso *dso, struct map *map,
 		  struct symsrc *syms_ss, struct symsrc *runtime_ss,
 		  symbol_filter_t filter, int kmodule)

@@ -938,7 +943,7 @@ new_symbol:
 		 * DWARF DW_compile_unit has this, but we don't always have access
 		 * to it...
 		 */
-		if (symbol_conf.demangle) {
+		if (want_demangle(dso->kernel || kmodule)) {
 			int demangle_flags = DMGL_NO_OPTS;
 			if (verbose)
 				demangle_flags = DMGL_PARAMS | DMGL_ANSI;

index ac098a3c2a31..1adb143867e3 100644
@@ -34,6 +34,7 @@ struct symbol_conf symbol_conf = {
 	.try_vmlinux_path	= true,
 	.annotate_src		= true,
 	.demangle		= true,
+	.demangle_kernel	= false,
 	.cumulate_callchain	= true,
 	.show_hist_headers	= true,
 	.symfs			= "",

index 3f95ea0357e3..bec4b7bd09de 100644
@@ -120,6 +120,7 @@ struct symbol_conf {
 			annotate_src,
 			event_group,
 			demangle,
+			demangle_kernel,
 			filter_relative,
 			show_hist_headers;
 	const char	*vmlinux_name,

1.9.3

From: John Spencer <[email protected]>

This makes it work with non-GNU grep's as well.

---
 tools/perf/config/utilities.mak | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 4d985e0f03f5..7076a62d0ff7 100644
@@ -132,7 +132,7 @@ endef
 #
 # Usage: bool-value = $(call is-absolute,path)
 #
-is-absolute = $(shell echo $(shell-sq) | grep ^/ -q && echo y)
+is-absolute = $(shell echo $(shell-sq) | grep -q ^/ && echo y)
 
 # lookup
 #

1.9.3

From: Arnaldo Carvalho de Melo <[email protected]>

Include poll.h instead.

Fixes the following warning in systems with musl's libc:

/usr/include/sys/poll.h:1:2: warning: #warning redirecting incorrect #include <sys/poll.h> to <poll.h> [-Wcpp]

---
 tools/perf/bench/sched-messaging.c | 2 +-
 tools/perf/builtin-top.c           | 2 +-
 tools/perf/util/util.h             | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

index 52a56599a543..d7f281c2828d 100644
@@ -26,7 +26,7 @@
 #include <sys/socket.h>
 #include <sys/wait.h>
 #include <sys/time.h>
-#include <sys/poll.h>
+#include <poll.h>
 #include <limits.h>
 #include <err.h>
 

index 7da2c46ea38f..e13864be2acb 100644
@@ -59,7 +59,7 @@
 
 #include <sys/syscall.h>
 #include <sys/ioctl.h>
-#include <sys/poll.h>
+#include <poll.h>
 #include <sys/prctl.h>
 #include <sys/wait.h>
 #include <sys/uio.h>

index d6a79b1fb28c..6ad2b5e3d5bb 100644
@@ -64,7 +64,7 @@
 #include <regex.h>
 #include <utime.h>
 #include <sys/wait.h>
-#include <sys/poll.h>
+#include <poll.h>
 #include <sys/socket.h>
 #include <sys/ioctl.h>
 #include <inttypes.h>

1.9.3

From: Chanho Park <[email protected]>

_BSD_SOURCE was deprecated in favour of _DEFAULT_SOURCE since glibc 2.20[1]. To avoid build warning on glibc2.20, _DEFAULT_SOURCE should also be defined.

[1]: https://sourceware.org/glibc/wiki/Release/2.20

---
 tools/perf/util/util.h | 2 ++
 1 file changed, 2 insertions(+)

index 6ad2b5e3d5bb..80bfdaa0e2a4 100644
@@ -39,6 +39,8 @@
 
 #define _ALL_SOURCE 1
 #define _BSD_SOURCE 1
+/* glibc 2.20 deprecates _BSD_SOURCE in favour of _DEFAULT_SOURCE */
+#define _DEFAULT_SOURCE 1
 #define HAS_BOOL
 
 #include <unistd.h>

1.9.3

From: Anton Blanchard <[email protected]>

If a vmlinux is stripped, perf will use it and ignore kallsyms. We end up with useless profiles where everything maps to a few runtime symbols:

    63.39%       swapper  [kernel.kallsyms]   [k] hcall_real_table
     4.90%      beam.smp  [kernel.kallsyms]   [k] hcall_real_table
     4.44%      beam.smp  [kernel.kallsyms]   [k] __sched_text_start
     3.72%      beam.smp  [kernel.kallsyms]   [k] __run_at_kexec

Detect this case and fallback to using kallsyms. This fixes the issue:

    62.81%       swapper  [kernel.kallsyms]   [k] snooze_loop
     4.44%      beam.smp  [kernel.kallsyms]   [k] __schedule
     0.91%      beam.smp  [kernel.kallsyms]   [k] _switch
     0.73%      beam.smp  [kernel.kallsyms]   [k] put_prev_entity

---
 tools/perf/util/symbol-elf.c | 8 ++++++++
 1 file changed, 8 insertions(+)

index 9c9b27fbc78d..2a92e10317c5 100644
@@ -717,6 +717,14 @@ int dso__load_sym(struct dso *dso, struct map *map,
 		symbols__delete(&dso->symbols[map->type]);
 
 	if (!syms_ss->symtab) {
+		/*
+		 * If the vmlinux is stripped, fail so we will fall back
+		 * to using kallsyms. The vmlinux runtime symbols aren't
+		 * of much use.
+		 */
+		if (dso->kernel)
+			goto out_elf_end;
+
 		syms_ss->symtab  = syms_ss->dynsym;
 		syms_ss->symshdr = syms_ss->dynshdr;
 	}

1.9.3

From: Anton Blanchard <[email protected]>

Ubuntu places the kernel debuginfo in /usr/lib/debug/boot/vmlinux-*

echo Link: http://lkml.kernel.org/n/tip-ranpwd -l 24@git.kernel.org

---
 tools/perf/util/symbol.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

index 1adb143867e3..be84f7a9838b 100644
@@ -1757,7 +1757,7 @@ static int vmlinux_path__init(struct perf_session_env *env)
 	char bf[PATH_MAX];
 	char *kernel_version;
 
-	vmlinux_path = malloc(sizeof(char *) * 5);
+	vmlinux_path = malloc(sizeof(char *) * 6);
 	if (vmlinux_path == NULL)
 		return -1;
 

@@ -1788,6 +1788,12 @@ static int vmlinux_path__init(struct perf_session_env *env)
 	if (vmlinux_path[vmlinux_path__nr_entries] == NULL)
 		goto out_fail;
 	++vmlinux_path__nr_entries;
+	snprintf(bf, sizeof(bf), "/usr/lib/debug/boot/vmlinux-%s",
+		 kernel_version);
+	vmlinux_path[vmlinux_path__nr_entries] = strdup(bf);
+	if (vmlinux_path[vmlinux_path__nr_entries] == NULL)
+		goto out_fail;
+        ++vmlinux_path__nr_entries;
 	snprintf(bf, sizeof(bf), "/lib/modules/%s/build/vmlinux", kernel_version);
 	vmlinux_path[vmlinux_path__nr_entries] = strdup(bf);
 	if (vmlinux_path[vmlinux_path__nr_entries] == NULL)

1.9.3

From: Masami Hiramatsu <[email protected]>

Do not access kallsyms to show available variables and show source lines in user binaries.

This behavior always requires the root privilege when sysctl sets kernel.kptr_restrict=1, but we don't need it just for analyzing user binaries.

Without this patch (by normal user, kptr_restrict=1):
----

  $ perf probe -x ./perf -V add_cmdname
  Failed to init vmlinux path.
    Error: Failed to show vars.
  $ perf probe -x ./perf -L add_cmdname
  Failed to init vmlinux path.
    Error: Failed to show lines.
  ----

With this patch:
----

  $ perf probe -x ./perf -V add_cmdname
  Available variables at add_cmdname
          @<perf_unknown_cmd_config+144>
                  (No matched variables)
          @<list_commands_in_dir+160>
                  (No matched variables)
          @<add_cmdname+0>
                  char*   name
                  size_t  len
                  struct cmdnames*        cmds
  $ perf probe -x ./perf -L add_cmdname
  <add_cmdname@/home/fedora/ksrc/linux-3/tools/perf/util/help.c:0>
        0  void add_cmdname(struct cmdnames *cmds, const char *name, size_t len)
        1  {
        2         struct cmdname *ent = malloc(sizeof(*ent) + len + 1);

        4         ent->len = len;
        5         memcpy(ent->name, name, len);
        6         ent->name[len] = 0;

...
----

---
 tools/perf/builtin-probe.c    | 3 ++-
 tools/perf/util/probe-event.c | 9 +++++----
 tools/perf/util/probe-event.h | 3 ++-
 3 files changed, 9 insertions(+), 6 deletions(-)

index 4d6858dbebea..04412b4770a2 100644
@@ -472,7 +472,8 @@ __cmd_probe(int argc, const char **argv, const char *prefix __maybe_unused)
 			usage_with_options(probe_usage, options);
 		}
 
-		ret = show_line_range(&params.line_range, params.target);
+		ret = show_line_range(&params.line_range, params.target,
+				      params.uprobes);
 		if (ret < 0)
 			pr_err_with_code("  Error: Failed to show lines.", ret);
 		return ret;

index f73595fc0627..be37b5aca335 100644
@@ -697,11 +697,11 @@ end:
 	return ret;
 }
 
-int show_line_range(struct line_range *lr, const char *module)
+int show_line_range(struct line_range *lr, const char *module, bool user)
 {
 	int ret;
 
-	ret = init_symbol_maps(false);
+	ret = init_symbol_maps(user);
 	if (ret < 0)
 		return ret;
 	ret = __show_line_range(lr, module);

@@ -776,7 +776,7 @@ int show_available_vars(struct perf_probe_event *pevs, int npevs,
 	int i, ret = 0;
 	struct debuginfo *dinfo;
 
-	ret = init_symbol_maps(false);
+	ret = init_symbol_maps(pevs->uprobes);
 	if (ret < 0)
 		return ret;
 

@@ -822,7 +822,8 @@ static int try_to_find_probe_trace_events(struct perf_probe_event *pev,
 }
 
 int show_line_range(struct line_range *lr __maybe_unused,
-		    const char *module __maybe_unused)
+		    const char *module __maybe_unused,
+		    bool user __maybe_unused)
 {
 	pr_warning("Debuginfo-analysis is not supported.\n");
 	return -ENOSYS;

index 776c9347a3b6..e01e9943139f 100644
@@ -128,7 +128,8 @@ extern int add_perf_probe_events(struct perf_probe_event *pevs, int npevs,
 				 bool force_add);
 extern int del_perf_probe_events(struct strlist *dellist);
 extern int show_perf_probe_events(void);
-extern int show_line_range(struct line_range *lr, const char *module);
+extern int show_line_range(struct line_range *lr, const char *module,
+			   bool user);
 extern int show_available_vars(struct perf_probe_event *pevs, int npevs,
 			       int max_probe_points, const char *module,
 			       struct strfilter *filter, bool externs);

1.9.3

From: Masami Hiramatsu <[email protected]>

Do not use dwfl_module_addrsym if dwarf_diename can find the symbol name, since dwfl_module_addrsym can be failed on shared libraries.

Without this patch
----

  $ perf probe -x ../lib/traceevent/libtraceevent.so -V create_arg_op
  Failed to find symbol at 0x11df1
  Failed to find the address of create_arg_op
    Error: Failed to show vars.
  ----
With this patch
  ----
  $ perf probe -x ../lib/traceevent/libtraceevent.so -V create_arg_op
  Available variables at create_arg_op
          @<create_arg_op+0>
                  enum filter_op_type     btype
                  struct filter_arg*      arg
  ----

This bug was reported on [email protected].

---
 tools/perf/util/probe-finder.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

index 9c593561aa71..c7918f83b300 100644
@@ -609,14 +609,18 @@ static int convert_to_trace_point(Dwarf_Die *sp_die, Dwfl_Module *mod,
 		return -EINVAL;
 	}
 
-	/* Get an appropriate symbol from symtab */
-	symbol = dwfl_module_addrsym(mod, paddr, &sym, NULL);
+	symbol = dwarf_diename(sp_die);
 	if (!symbol) {
-		pr_warning("Failed to find symbol at 0x%lx\n",
-			   (unsigned long)paddr);
-		return -ENOENT;
+		/* Try to get the symbol name from symtab */
+		symbol = dwfl_module_addrsym(mod, paddr, &sym, NULL);
+		if (!symbol) {
+			pr_warning("Failed to find symbol at 0x%lx\n",
+				   (unsigned long)paddr);
+			return -ENOENT;
+		}
+		eaddr = sym.st_value;
 	}
-	tp->offset = (unsigned long)(paddr - sym.st_value);
+	tp->offset = (unsigned long)(paddr - eaddr);
 	tp->address = (unsigned long)paddr;
 	tp->symbol = strdup(symbol);
 	if (!tp->symbol)

1.9.3

From: Arnaldo Carvalho de Melo <[email protected]>

All builtins that consume events from perf's ring buffer now end up calling perf_evlist__mmap_consume(), which will allow unmapping the ring buffer when all the fds gets closed and all events in the buffer consumed.

This is in preparation for the patchkit that will notice POLLHUP on perf events file descriptors.

---
 tools/perf/builtin-record.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

index 87e28a4e33ba..a1b040394170 100644
@@ -65,8 +65,9 @@ static int process_synthesized_event(struct perf_tool *tool,
 	return record__write(rec, event, event->header.size);
 }
 
-static int record__mmap_read(struct record *rec, struct perf_mmap *md)
+static int record__mmap_read(struct record *rec, int idx)
 {
+	struct perf_mmap *md = &rec->evlist->mmap[idx];
 	unsigned int head = perf_mmap__read_head(md);
 	unsigned int old = md->prev;
 	unsigned char *data = md->base + page_size;

@@ -102,8 +103,7 @@ static int record__mmap_read(struct record *rec, struct perf_mmap *md)
 	}
 
 	md->prev = old;
-	perf_mmap__write_tail(md, old);
-
+	perf_evlist__mmap_consume(rec->evlist, idx);
 out:
 	return rc;
 }

@@ -245,7 +245,7 @@ static int record__mmap_read_all(struct record *rec)
 
 	for (i = 0; i < rec->evlist->nr_mmaps; i++) {
 		if (rec->evlist->mmap[i].base) {
-			if (record__mmap_read(rec, &rec->evlist->mmap[i]) != 0) {
+			if (record__mmap_read(rec, i) != 0) {
 				rc = -1;
 				goto out;
 			}

1.9.3

Pulled into tip:perf/core, thanks a lot Arnaldo!

Ingo