perf/core improvements and fixes

Hi Ingo,

Please consider pulling,

- Arnaldo

The following changes since commit d3aaf09f889b31f3b424bf9603b163ec1204c361:

Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-02-04 08:58:01 +0100)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo

for you to fetch changes up to 598b7c6919c7bbcc1243009721a01bc12275ff3e:

perf jit: add source line info support (2016-02-05 12:33:09 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

User visible fixes:

- Handle spaces in file names obtained from /proc/pid/maps (Marcin Ślusarz)

New features:

- Improved support for java, using the JVMTI agent library to do jitdumps that then will be inserted in synthesized PERF_RECORD_MMAP2 events via 'perf inject' pointed to synthesized ELF files stored in ~/.debug and keyed with build-ids, to allow symbol resolution and even annotation with source line info, see the changeset comments to see how to use it (Stephane Eranian)

Documentation:

- Document mmore variables in the 'perf config' man page (Taeung Song)

Infrastructure:

- Improve a bit the 'make -C tools/perf build-test' output (Arnaldo Carvalho de Melo)

- Do 'build-test' in parallell, using 'make -j' (Arnaldo Carvalho de Melo)

- Fix handling of 'clean' in multi-target make invokations for parallell builds (Jiri Olsa)

----------------------------------------------------------------
Arnaldo Carvalho de Melo (4):
perf build tests: Elide "-f Makefile" from make invokation
perf build tests: Move the feature related vars to the front of the make cmdline perf build tests: Do parallell builds with 'build-test'
perf inject: Make sure mmap records are ordered when injecting build_ids

Jiri Olsa (1):
perf tools: Fix parallel build including 'clean' target

Marcin Ślusarz (1):
perf tools: handle spaces in file names obtained from /proc/pid/maps

Stephane Eranian (5):
perf symbols: add Java demangling support perf build: Add libcrypto feature detection perf inject: Add jitdump mmap injection support perf tools: add JVMTI agent library perf jit: add source line info support

Taeung Song (8):
perf config: Document 'ui.show-headers' variable in man page perf config: Document variables for 'call-graph' section in man page perf config: Document variables for 'report' section in man page perf config: Document 'top.children' variable in man page perf config: Document 'man.viewer' variable in man page perf config: Document 'pager.<subcommand>' variables in man page perf config: Document 'kmem.default' variable in man page perf config: Document 'record.build-id' variable in man page

 tools/build/Makefile.feature             |   2 +
 tools/build/feature/Makefile             |   4 +
 tools/build/feature/test-all.c           |   5 +
 tools/build/feature/test-libcrypto.c     |  17 +
 tools/perf/Documentation/perf-config.txt | 143 +++++++
 tools/perf/Documentation/perf-inject.txt |   7 +
 tools/perf/Makefile                      |  16 +-
 tools/perf/Makefile.perf                 |   3 +
 tools/perf/builtin-inject.c              | 107 ++++-
 tools/perf/config/Makefile               |  11 +
 tools/perf/jvmti/Makefile                |  76 ++++
 tools/perf/jvmti/jvmti_agent.c           | 465 +++++++++++++++++++++
 tools/perf/jvmti/jvmti_agent.h           |  36 ++
 tools/perf/jvmti/libjvmti.c              | 304 ++++++++++++++
 tools/perf/tests/make                    |  11 +-
 tools/perf/util/Build                    |   6 +
 tools/perf/util/demangle-java.c          | 199 +++++++++
 tools/perf/util/demangle-java.h          |  10 +
 tools/perf/util/event.c                  |   2 +-
 tools/perf/util/genelf.c                 | 449 +++++++++++++++++++++
 tools/perf/util/genelf.h                 |  67 +++
 tools/perf/util/genelf_debug.c           | 610 ++++++++++++++++++++++++++++
 tools/perf/util/jit.h                    |  15 +
 tools/perf/util/jitdump.c                | 672 +++++++++++++++++++++++++++++++
 tools/perf/util/jitdump.h                | 124 ++++++
 tools/perf/util/symbol-elf.c             |   3 +
 26 files changed, 3357 insertions(+), 7 deletions(-)
 create mode 100644 tools/build/feature/test-libcrypto.c
 create mode 100644 tools/perf/jvmti/Makefile
 create mode 100644 tools/perf/jvmti/jvmti_agent.c
 create mode 100644 tools/perf/jvmti/jvmti_agent.h
 create mode 100644 tools/perf/jvmti/libjvmti.c
 create mode 100644 tools/perf/util/demangle-java.c
 create mode 100644 tools/perf/util/demangle-java.h
 create mode 100644 tools/perf/util/genelf.c
 create mode 100644 tools/perf/util/genelf.h
 create mode 100644 tools/perf/util/genelf_debug.c
 create mode 100644 tools/perf/util/jit.h
 create mode 100644 tools/perf/util/jitdump.c
 create mode 100644 tools/perf/util/jitdump.h

See http://www.infradead.org/rpr.html

From: Arnaldo Carvalho de Melo <[email protected]>

Since this is the name that 'make' will look for if no explicit -f file is passed.

This in turn makes the output of 'build-test' more compact:

Before:

  $ perf stat make -C tools/perf build-test
  <SNIP>
  cd . && make FEATURE_DUMP_COPY=/home/acme/git/linux/tools/perf/BUILD_TEST_FEATURE_DUMP feature-dump
          make_no_libaudit_O: cd . && make -f Makefile  O=/tmp/tmp.tHIa0Kkk2Y DESTDIR=/tmp/tmp.foK7rckkVi NO_LIBAUDIT=1 FEATURES_DUMP=/home/acme/git/linux/tools/perf/BUILD_TEST_FEATURE_DUMP
  <SNIP>

After:

  $ perf stat make -C tools/perf build-test
  <SNIP>
          make_no_libaudit_O: cd . && make  O=/tmp/tmp.tHIa0Kkk2Y DESTDIR=/tmp/tmp.foK7rckkVi NO_LIBAUDIT=1 FEATURES_DUMP=/home/acme/git/linux/tools/perf/BUILD_TEST_FEATURE_DUMP
  <SNIP>

---
 tools/perf/tests/make | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

index cc72b67bde5e..0b70cf16a562 100644
@@ -111,6 +111,9 @@ run := make_pure
 # disable features detection
 ifeq ($(MK),Makefile)
 run += make_clean_all
+MAKE_F := $(MAKE)
+else
+MAKE_F := $(MAKE) -f $(MK)
 endif
 run += make_python_perf_so
 run += make_debug

@@ -270,12 +273,12 @@ endif
 
 MAKEFLAGS := --no-print-directory
 
-clean := @(cd $(PERF); make -s -f $(MK) $(O_OPT) clean >/dev/null)
+clean := @(cd $(PERF); $(MAKE_F) -s $(O_OPT) clean >/dev/null)
 
 $(run):
 	$(call clean)
 	@TMP_DEST=$$(mktemp -d); \
-	cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
+	cmd="cd $(PERF) && $(MAKE_F) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
 	printf "%*.*s: %s\n" $(max_width) $(max_width) "$@" "$$cmd" && echo $$cmd > $@ && \
 	( eval $$cmd ) >> $@ 2>&1; \
 	echo "  test: $(call test,$@)" >> $@ 2>&1; \

@@ -286,7 +289,7 @@ $(run_O):
 	$(call clean)
 	@TMP_O=$$(mktemp -d); \
 	TMP_DEST=$$(mktemp -d); \
-	cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
+	cmd="cd $(PERF) && $(MAKE_F) $(PARALLEL_OPT) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
 	printf "%*.*s: %s\n" $(max_width) $(max_width) "$@" "$$cmd" && echo $$cmd > $@ && \
 	( eval $$cmd ) >> $@ 2>&1 && \
 	echo "  test: $(call test_O,$@)" >> $@ 2>&1; \

2.5.0

From: Arnaldo Carvalho de Melo <[email protected]>

So that we do less visual searching on the 'make build-test' output to see the feature related variables:

After:

  $ make -C tools/perf build-test
  <SNIP>
           make_no_newt_O: cd . && make NO_NEWT=1 FEATURES_DUMP=/home/acme/git/linux/tools/perf/BUILD_TEST_FEATURE_DUMP O=/tmp/tmp.dz55IX DESTDIR=/tmp/tmp.X29xxo
              make_tags_O: cd . && make tags FEATURES_DUMP=/home/acme/git/linux/tools/perf/BUILD_TEST_FEATURE_DUMP O=/tmp/tmp.6ecLh8 DESTDIR=/tmp/tmp.6vIla578Ho
  make_util_pmu_bison_o_O: cd . && make util/pmu-bison.o FEATURES_DUMP=/home/acme/git/linux/tools/perf/BUILD_TEST_FEATURE_DUMP O=/tmp/tmp.SVPM2G DESTDIR=/tmp/tmp.C0oAam

---
 tools/perf/tests/make | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 0b70cf16a562..12dcae7aa515 100644
@@ -278,7 +278,7 @@ clean := @(cd $(PERF); $(MAKE_F) -s $(O_OPT) clean >/dev/null)
 $(run):
 	$(call clean)
 	@TMP_DEST=$$(mktemp -d); \
-	cmd="cd $(PERF) && $(MAKE_F) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
+	cmd="cd $(PERF) && $(MAKE_F) $($@) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST"; \
 	printf "%*.*s: %s\n" $(max_width) $(max_width) "$@" "$$cmd" && echo $$cmd > $@ && \
 	( eval $$cmd ) >> $@ 2>&1; \
 	echo "  test: $(call test,$@)" >> $@ 2>&1; \

@@ -289,7 +289,7 @@ $(run_O):
 	$(call clean)
 	@TMP_O=$$(mktemp -d); \
 	TMP_DEST=$$(mktemp -d); \
-	cmd="cd $(PERF) && $(MAKE_F) $(PARALLEL_OPT) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
+	cmd="cd $(PERF) && $(MAKE_F) $($(patsubst %_O,%,$@)) $(PARALLEL_OPT) O=$$TMP_O DESTDIR=$$TMP_DEST"; \
 	printf "%*.*s: %s\n" $(max_width) $(max_width) "$@" "$$cmd" && echo $$cmd > $@ && \
 	( eval $$cmd ) >> $@ 2>&1 && \
 	echo "  test: $(call test_O,$@)" >> $@ 2>&1; \

2.5.0

See http://www.infradead.org/rpr.html

From: Taeung Song <[email protected]>

This option controls display of column headers (like 'Overhead' and 'Symbol') in 'report' and 'top'. If this option is false, they are hidden. This option is only applied to TUI.

---
 tools/perf/Documentation/perf-config.txt | 6 ++++++
 1 file changed, 6 insertions(+)

index 74589c68558a..42787222ad15 100644
@@ -296,6 +296,12 @@ hist.*::
 	       and 'baz' to 50.00% for each, while 'absolute' would show their
 	       current overhead (33.33%).
 
+ui.*::
+	ui.show-headers::
+		This option controls display of column headers (like 'Overhead' and 'Symbol')
+		in 'report' and 'top'. If this option is false, they are hidden.
+		This option is only applied to TUI.
+
 SEE ALSO
 --------
 linkperf:perf[1]

2.5.0

See http://www.infradead.org/rpr.html

From: Taeung Song <[email protected]>

Explain 'call-graph' section and its variables:

'record-mode', 'dump-size', 'print-type', 'order', 'sort-key', 'threshold' and 'print-limit'.

---
 tools/perf/Documentation/perf-config.txt | 67 ++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

index 42787222ad15..42310ae7e636 100644
@@ -302,6 +302,73 @@ ui.*::
 		in 'report' and 'top'. If this option is false, they are hidden.
 		This option is only applied to TUI.
 
+call-graph.*::
+	When sub-commands 'top' and 'report' work with -g/—-children
+	there're options in control of call-graph.
+
+	call-graph.record-mode::
+		The record-mode can be 'fp' (frame pointer), 'dwarf' and 'lbr'.
+		The value of 'dwarf' is effective only if perf detect needed library
+		(libunwind or a recent version of libdw).
+		'lbr' only work for cpus that support it.
+
+	call-graph.dump-size::
+		The size of stack to dump in order to do post-unwinding. Default is 8192 (byte).
+		When using dwarf into record-mode, the default size will be used if omitted.
+
+	call-graph.print-type::
+		The print-types can be graph (graph absolute), fractal (graph relative),
+		flat and folded. This option controls a way to show overhead for each callchain
+		entry. Suppose a following example.
+
+                Overhead  Symbols
+                ........  .......
+                  40.00%  foo
+                          |
+                          ---foo
+                             |
+                             |--50.00%--bar
+                             |          main
+                             |
+                              --50.00%--baz
+                                        main
+
+		This output is a 'fractal' format. The 'foo' came from 'bar' and 'baz' exactly
+		half and half so 'fractal' shows 50.00% for each
+		(meaning that it assumes 100% total overhead of 'foo').
+
+		The 'graph' uses absolute overhead value of 'foo' as total so each of
+		'bar' and 'baz' callchain will have 20.00% of overhead.
+		If 'flat' is used, single column and linear exposure of call chains.
+		'folded' mean call chains are displayed in a line, separated by semicolons.
+
+	call-graph.order::
+		This option controls print order of callchains. The default is
+		'callee' which means callee is printed at top and then followed by its
+		caller and so on. The 'caller' prints it in reverse order.
+
+		If this option is not set and report.children or top.children is
+		set to true (or the equivalent command line option is given),
+		the default value of this option is changed to 'caller' for the
+		execution of 'perf report' or 'perf top'. Other commands will
+		still default to 'callee'.
+
+	call-graph.sort-key::
+		The callchains are merged if they contain same information.
+		The sort-key option determines a way to compare the callchains.
+		A value of 'sort-key' can be 'function' or 'address'.
+		The default is 'function'.
+
+	call-graph.threshold::
+		When there're many callchains it'd print tons of lines. So perf omits
+		small callchains under a certain overhead (threshold) and this option
+		control the threshold. Default is 0.5 (%). The overhead is calculated
+		by value depends on call-graph.print-type.
+
+	call-graph.print-limit::
+		This is a maximum number of lines of callchain printed for a single
+		histogram entry. Default is 0 which means no limitation.
+
 SEE ALSO
 --------
 linkperf:perf[1]

2.5.0

See http://www.infradead.org/rpr.html

From: Taeung Song <[email protected]>

Explain 'report' section's variables:

'percent-limit', 'queue-size' and 'children'.

---
 tools/perf/Documentation/perf-config.txt | 36 ++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

index 42310ae7e636..f38f46f67d74 100644
@@ -369,6 +369,42 @@ call-graph.*::
 		This is a maximum number of lines of callchain printed for a single
 		histogram entry. Default is 0 which means no limitation.
 
+report.*::
+	report.percent-limit::
+		This one is mostly the same as call-graph.threshold but works for
+		histogram entries. Entries having an overhead lower than this
+		percentage will not be printed. Default is '0'. If percent-limit
+		is '10', only entries which have more than 10% of overhead will be
+		printed.
+
+	report.queue-size::
+		This option sets up the maximum allocation size of the internal
+		event queue for ordering events. Default is 0, meaning no limit.
+
+	report.children::
+		'Children' means functions called from another function.
+		If this option is true, 'perf report' cumulates callchains of children
+		and show (accumulated) total overhead as well as 'Self' overhead.
+		Please refer to the 'perf report' manual. The default is 'true'.
+
+	report.group::
+		This option is to show event group information together.
+		Example output with this turned on, notice that there is one column
+		per event in the group, ref-cycles and cycles:
+
+		# group: {ref-cycles,cycles}
+		# ========
+		#
+		# Samples: 7K of event 'anon group { ref-cycles, cycles }'
+		# Event count (approx.): 6876107743
+		#
+		#         Overhead  Command      Shared Object               Symbol
+		# ................  .......  .................  ...................
+		#
+		    99.84%  99.76%  noploop  noploop            [.] main
+		     0.07%   0.00%  noploop  ld-2.15.so         [.] strcmp
+		     0.03%   0.00%  noploop  [kernel.kallsyms]  [k] timerqueue_del
+
 SEE ALSO
 --------
 linkperf:perf[1]

2.5.0

From: Taeung Song <[email protected]>

Explain 'top.children' variable.

---
 tools/perf/Documentation/perf-config.txt | 7 +++++++
 1 file changed, 7 insertions(+)

index f38f46f67d74..5e1db5ae53c4 100644
@@ -405,6 +405,13 @@ report.*::
 		     0.07%   0.00%  noploop  ld-2.15.so         [.] strcmp
 		     0.03%   0.00%  noploop  [kernel.kallsyms]  [k] timerqueue_del
 
+top.*::
+	top.children::
+		Same as 'report.children'. So if it is enabled, the output of 'top'
+		command will have 'Children' overhead column as well as 'Self' overhead
+		column by default.
+		The default is 'true'.
+
 SEE ALSO
 --------
 linkperf:perf[1]

2.5.0

From: Taeung Song <[email protected]>

Explain 'man.viewer' variable and how to add new man viewer tools.

---
 tools/perf/Documentation/perf-config.txt | 9 +++++++++
 1 file changed, 9 insertions(+)

index 5e1db5ae53c4..fd3f048c9644 100644
@@ -412,6 +412,15 @@ top.*::
 		column by default.
 		The default is 'true'.
 
+man.*::
+	man.viewer::
+		This option can assign a tool to view manual pages when 'help'
+		subcommand was invoked. Supported tools are 'man', 'woman'
+		(with emacs client) and 'konqueror'. Default is 'man'.
+
+		New man viewer tool can be also added using 'man.<tool>.cmd'
+		or use different path using 'man.<tool>.path' config option.
+
 SEE ALSO
 --------
 linkperf:perf[1]

2.5.0

See http://www.infradead.org/rpr.html

From: Taeung Song <[email protected]>

Explain 'pager.<subcommand>' variables.

---
 tools/perf/Documentation/perf-config.txt | 5 +++++
 1 file changed, 5 insertions(+)

index fd3f048c9644..99aa72e5e9cf 100644
@@ -421,6 +421,11 @@ man.*::
 		New man viewer tool can be also added using 'man.<tool>.cmd'
 		or use different path using 'man.<tool>.path' config option.
 
+pager.*::
+	pager.<subcommand>::
+		When the subcommand is run on stdio, determine whether it uses
+		pager or not based on this value. Default is 'unspecified'.
+
 SEE ALSO
 --------
 linkperf:perf[1]

2.5.0

From: Taeung Song <[email protected]>

Explain 'kmem.default' variable.

---
 tools/perf/Documentation/perf-config.txt | 5 +++++
 1 file changed, 5 insertions(+)

index 99aa72e5e9cf..fb1f4a984e63 100644
@@ -426,6 +426,11 @@ pager.*::
 		When the subcommand is run on stdio, determine whether it uses
 		pager or not based on this value. Default is 'unspecified'.
 
+kmem.*::
+	kmem.default::
+		This option decides which allocator is to be analyzed if neither
+		'--slab' nor '--page' option is used. Default is 'slab'.
+
 SEE ALSO
 --------
 linkperf:perf[1]

2.5.0

See http://www.infradead.org/rpr.html

From: Taeung Song <[email protected]>

Explain 'record.build-id' variable.

---
 tools/perf/Documentation/perf-config.txt | 8 ++++++++
 1 file changed, 8 insertions(+)

index fb1f4a984e63..c7158bfb1649 100644
@@ -431,6 +431,14 @@ kmem.*::
 		This option decides which allocator is to be analyzed if neither
 		'--slab' nor '--page' option is used. Default is 'slab'.
 
+record.*::
+	record.build-id::
+		This option can be 'cache', 'no-cache' or 'skip'.
+		'cache' is to post-process data and save/update the binaries into
+		the build-id cache (in ~/.debug). This is the default.
+		But if this option is 'no-cache', it will not update the build-id cache.
+		'skip' skips post-processing and does not update the cache.
+
 SEE ALSO
 --------
 linkperf:perf[1]

2.5.0

From: Jiri Olsa <[email protected]>

Do not parallelize 'clean' with other targets, figure out if it is present and do it first, then the other targets.

Noticed with:

tools/perf> make -j24 clean all

LD arch/libperf-in.o
LD plugin_xen-in.o
arch//libperf-in.o: file not recognized: File truncated make[3]: *** [arch/libperf-in.o] Error 1 make[2]: *** [arch] Error 2
make[2]: *** Waiting for unfinished jobs....
AR libapi.a

---
 tools/perf/Makefile | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

index 4b68f465195c..67837c6cdbd8 100644
@@ -68,6 +68,20 @@ all tags TAGS:
 	$(print_msg)
 	$(make)
 
+ifdef MAKECMDGOALS
+has_clean := 0
+ifneq ($(filter clean,$(MAKECMDGOALS)),)
+  has_clean := 1
+endif # clean
+
+ifeq ($(has_clean),1)
+  rest := $(filter-out clean,$(MAKECMDGOALS))
+  ifneq ($(rest),)
+$(rest): clean
+  endif # rest
+endif # has_clean
+endif # MAKECMDGOALS
+
 #
 # The clean target is not really parallel, don't print the jobs info:
 #

2.5.0

See http://www.infradead.org/rpr.html

From: Arnaldo Carvalho de Melo <[email protected]>

---
 tools/perf/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 67837c6cdbd8..32a64e619028 100644
@@ -99,7 +99,7 @@ clean:
 # make -C tools/perf -f tests/make
 #
 build-test:
-	@$(MAKE) SHUF=1 -f tests/make REUSE_FEATURES_DUMP=1 MK=Makefile --no-print-directory tarpkg out
+	@$(MAKE) SHUF=1 -f tests/make REUSE_FEATURES_DUMP=1 MK=Makefile SET_PARALLEL=1 --no-print-directory tarpkg out
 
 #
 # All other targets get passed through:

2.5.0

From: Marcin Ślusarz <[email protected]>

Steam frequently puts game binaries in folders with spaces.

Note: "(deleted)" markers are now treated as part of the file name.

---
 tools/perf/util/event.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 85155e91b61b..7bad5c3fa7b7 100644
@@ -282,7 +282,7 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,
 		strcpy(execname, "");
 
 		/* 00400000-0040c000 r-xp 00000000 fd:01 41038  /bin/cat */
-		n = sscanf(bf, "%"PRIx64"-%"PRIx64" %s %"PRIx64" %x:%x %u %s\n",
+		n = sscanf(bf, "%"PRIx64"-%"PRIx64" %s %"PRIx64" %x:%x %u %[^\n]\n",
 		       &event->mmap2.start, &event->mmap2.len, prot,
 		       &event->mmap2.pgoff, &event->mmap2.maj,
 		       &event->mmap2.min,

2.5.0

See http://www.infradead.org/rpr.html

From: Stephane Eranian <[email protected]>

Add Java function descriptor demangling support. Something bfd cannot do.

Use the JAVA_DEMANGLE_NORET flag to avoid decoding the return type of functions.

---
 tools/perf/util/Build           |   1 +
 tools/perf/util/demangle-java.c | 199 ++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/demangle-java.h |  10 ++
 tools/perf/util/symbol-elf.c    |   3 +
 4 files changed, 213 insertions(+)
 create mode 100644 tools/perf/util/demangle-java.c
 create mode 100644 tools/perf/util/demangle-java.h

index 5eec53a3f4ac..edae107416b6 100644
@@ -105,6 +105,7 @@ libperf-y += scripting-engines/
 
 libperf-$(CONFIG_ZLIB) += zlib.o
 libperf-$(CONFIG_LZMA) += lzma.o
+libperf-y += demangle-java.o
 
 CFLAGS_config.o   += -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
 

new file mode 100644
index 000000000000..a981c1f968fe
@@ -0,0 +1,10 @@
+#ifndef __PERF_DEMANGLE_JAVA
+#define __PERF_DEMANGLE_JAVA 1
+/*
+ * demangle function flags
+ */
+#define JAVA_DEMANGLE_NORET	0x1 /* do not process return type */
+
+char * java_demangle_sym(const char *str, int flags);
+
+#endif /* __PERF_DEMANGLE_JAVA */

index 562b8ebeae5b..b1dd68f358fc 100644
@@ -6,6 +6,7 @@
 #include <inttypes.h>
 
 #include "symbol.h"
+#include "demangle-java.h"
 #include "machine.h"
 #include "vdso.h"
 #include <symbol/kallsyms.h>

@@ -1077,6 +1078,8 @@ new_symbol:
 				demangle_flags = DMGL_PARAMS | DMGL_ANSI;
 
 			demangled = bfd_demangle(NULL, elf_name, demangle_flags);
+			if (demangled == NULL)
+				demangled = java_demangle_sym(elf_name, JAVA_DEMANGLE_NORET);
 			if (demangled != NULL)
 				elf_name = demangled;
 		}

2.5.0

From: Stephane Eranian <[email protected]>

Will be used to generate build-ids in the jitdump code.

---
 tools/build/Makefile.feature         |  2 ++
 tools/build/feature/Makefile         |  4 ++++
 tools/build/feature/test-all.c       |  5 +++++
 tools/build/feature/test-libcrypto.c | 17 +++++++++++++++++
 tools/perf/Makefile.perf             |  3 +++
 tools/perf/config/Makefile           | 11 +++++++++++
 tools/perf/tests/make                |  2 ++
 7 files changed, 44 insertions(+)
 create mode 100644 tools/build/feature/test-libcrypto.c

index 7bff2ea831cf..6b7707270aa3 100644
@@ -46,6 +46,7 @@ FEATURE_TESTS_BASIC :=			\
 	libpython			\
 	libpython-version		\
 	libslang			\
+	libcrypto			\
 	libunwind			\
 	pthread-attr-setaffinity-np	\
 	stackprotector-all		\

@@ -87,6 +88,7 @@ FEATURE_DISPLAY ?=			\
 	libperl				\
 	libpython			\
 	libslang			\
+	libcrypto			\
 	libunwind			\
 	libdw-dwarf-unwind		\
 	zlib				\

index bf8f0352264d..c5f4c417428d 100644
@@ -23,6 +23,7 @@ FILES=					\
 	test-libpython.bin		\
 	test-libpython-version.bin	\
 	test-libslang.bin		\
+	test-libcrypto.bin		\
 	test-libunwind.bin		\
 	test-libunwind-debug-frame.bin	\
 	test-pthread-attr-setaffinity-np.bin	\

@@ -105,6 +106,9 @@ $(OUTPUT)test-libaudit.bin:
 $(OUTPUT)test-libslang.bin:
 	$(BUILD) -I/usr/include/slang -lslang
 
+$(OUTPUT)test-libcrypto.bin:
+	$(BUILD) -lcrypto
+
 $(OUTPUT)test-gtk2.bin:
 	$(BUILD) $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null)
 

index 81025cade45f..e499a36c1e4a 100644
@@ -129,6 +129,10 @@
 # include "test-bpf.c"
 #undef main
 
+#define main main_test_libcrypto
+# include "test-libcrypto.c"
+#undef main
+
 int main(int argc, char *argv[])
 {
 	main_test_libpython();

@@ -158,6 +162,7 @@ int main(int argc, char *argv[])
 	main_test_lzma();
 	main_test_get_cpuid();
 	main_test_bpf();
+	main_test_libcrypto();
 
 	return 0;
 }

new file mode 100644
index 000000000000..bd79dc7f28d3
@@ -0,0 +1,17 @@
+#include <openssl/sha.h>
+#include <openssl/md5.h>
+
+int main(void)
+{
+	MD5_CTX context;
+	unsigned char md[MD5_DIGEST_LENGTH + SHA_DIGEST_LENGTH];
+	unsigned char dat[] = "12345";
+
+	MD5_Init(&context);
+	MD5_Update(&context, &dat[0], sizeof(dat));
+	MD5_Final(&md[0], &context);
+
+	SHA1(&dat[0], sizeof(dat), &md[0]);
+
+	return 0;
+}

index 0ef3d97d7954..d404117810a7 100644
@@ -58,6 +58,9 @@ include config/utilities.mak
 #
 # Define NO_LIBBIONIC if you do not want bionic support
 #
+# Define NO_LIBCRYPTO if you do not want libcrypto (openssl) support
+# used for generating build-ids for ELFs generated by jitdump.
+#
 # Define NO_LIBDW_DWARF_UNWIND if you do not want libdw support
 # for dwarf backtrace post unwind.
 #

index 0045a5ddd0ca..f7aeaf303f5a 100644
@@ -404,6 +404,17 @@ ifndef NO_LIBAUDIT
   endif
 endif
 
+ifndef NO_LIBCRYPTO
+  ifneq ($(feature-libcrypto), 1)
+    msg := $(warning No libcrypto.h found, disables jitted code injection, please install libssl-devel or libssl-dev);
+    NO_LIBCRYPTO := 1
+  else
+    CFLAGS += -DHAVE_LIBCRYPTO_SUPPORT
+    EXTLIBS += -lcrypto
+    $(call detected,CONFIG_CRYPTO)
+  endif
+endif
+
 ifdef NO_NEWT
   NO_SLANG=1
 endif

index 12dcae7aa515..cac15d93aea6 100644
@@ -80,6 +80,7 @@ make_no_libaudit    := NO_LIBAUDIT=1
 make_no_libbionic   := NO_LIBBIONIC=1
 make_no_auxtrace    := NO_AUXTRACE=1
 make_no_libbpf	    := NO_LIBBPF=1
+make_no_libcrypto   := NO_LIBCRYPTO=1
 make_tags           := tags
 make_cscope         := cscope
 make_help           := help

@@ -103,6 +104,7 @@ make_minimal        := NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1
 make_minimal        += NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1 NO_BACKTRACE=1
 make_minimal        += NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1
 make_minimal        += NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1 NO_LIBBPF=1
+make_minimal        += NO_LIBCRYPTO=1
 
 # $(run) contains all available tests
 run := make_pure

2.5.0

From: Arnaldo Carvalho de Melo <[email protected]>

To make sure the mmap records are ordered correctly and so that the correct especially due to jitted code mmaps.

We cannot generate the buildid hit list and inject the jit mmaps (will come right after this patch) in at the same time for now.

---
 tools/perf/builtin-inject.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

index 0022e02ed31a..6567baedd92a 100644
@@ -755,6 +755,17 @@ int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
 	if (inject.session == NULL)
 		return -1;
 
+	if (inject.build_ids) {
+		/*
+		 * to make sure the mmap records are ordered correctly
+		 * and so that the correct especially due to jitted code
+		 * mmaps. We cannot generate the buildid hit list and
+		 * inject the jit mmaps at the same time for now.
+		 */
+		inject.tool.ordered_events = true;
+		inject.tool.ordering_requires_timestamps = true;
+	}
+
 	ret = symbol__init(&inject.session->header.env);
 	if (ret < 0)
 		goto out_delete;

2.5.0

From: Stephane Eranian <[email protected]>

This patch adds a --jit/-j option to perf inject.

This options injects MMAP records into the perf.data file to cover the jitted code mmaps. It also emits ELF images for each function in the jidump file. Those images are created where the jitdump file is. The MMAP records point to that location as well.

Typical flow:

  $ perf record -k mono -- java -agentpath:libpjvmti.so java_class
  $ perf inject --jit -i perf.data -o perf.data.jitted
  $ perf report -i perf.data.jitted

Note that jitdump.h support is not limited to Java, it works with any jitted environment modified to emit the jitdump file format, include those where code can be jitted multiple times and moved around.

The jitdump.h format is adapted from the Oprofile project.

The genelf.c (ELF binary generation) depends on MD5 hash encoding for the buildid. To enable this, libssl-dev must be installed. If not, then genelf.c defaults to using urandom to generate the buildid, which is not ideal. The Makefile auto-detects the presence on libssl-dev.

This version mmaps the jitdump file to create a marker MMAP record in the perf.data file. The marker is used to detect jitdump and cause perf inject to inject the jitted mmaps and generate ELF images for jitted functions.

In V8, the following fixes and changes were made among other things:

- the jidump header format include a new flags field to be used to carry information about the configuration of the runtime agent. Contributed by: Adrian Hunter <[email protected]>

- Fix mmap pgoff: MMAP event pgoff must be the offset within the ELF file at which the code resides.
Contributed by: Adrian Hunter <[email protected]>

- Fix ELF virtual addresses: perf tools expect the ELF virtual addresses of dynamic objects to match the file offset.
Contributed by: Adrian Hunter <[email protected]>

- JIT MMAP injection does not obey finished_round semantics. JIT MMAP injection injects all MMAP events in one go, so it does not obey finished_round semantics, so drop the finished_round events from the output perf.data file.
Contributed by: Adrian Hunter <[email protected]>

---
 tools/perf/Documentation/perf-inject.txt |   7 +
 tools/perf/builtin-inject.c              |  98 ++++-
 tools/perf/util/Build                    |   2 +
 tools/perf/util/genelf.c                 | 442 ++++++++++++++++++++
 tools/perf/util/genelf.h                 |  63 +++
 tools/perf/util/jit.h                    |  15 +
 tools/perf/util/jitdump.c                | 670 +++++++++++++++++++++++++++++++
 tools/perf/util/jitdump.h                | 124 ++++++
 8 files changed, 1418 insertions(+), 3 deletions(-)
 create mode 100644 tools/perf/util/genelf.c
 create mode 100644 tools/perf/util/genelf.h
 create mode 100644 tools/perf/util/jit.h
 create mode 100644 tools/perf/util/jitdump.c
 create mode 100644 tools/perf/util/jitdump.h

index 0b1cedeef895..87b2588d1cbd 100644
@@ -53,6 +53,13 @@ include::itrace.txt[]
 --strip::
 	Use with --itrace to strip out non-synthesized events.
 
+-j::
+--jit::
+	Process jitdump files by injecting the mmap records corresponding to jitted
+	functions. This option also generates the ELF images for each jitted function
+	found in the jitdumps files captured in the input perf.data file. Use this option
+	if you are monitoring environment using JIT runtimes, such as Java, DART or V8.
+
 SEE ALSO
 --------
 linkperf:perf-record[1], linkperf:perf-report[1], linkperf:perf-archive[1]

index edae107416b6..52a4a806ee2f 100644
@@ -106,6 +106,8 @@ libperf-y += scripting-engines/
 libperf-$(CONFIG_ZLIB) += zlib.o
 libperf-$(CONFIG_LZMA) += lzma.o
 libperf-y += demangle-java.o
+libperf-$(CONFIG_LIBELF) += jitdump.o
+libperf-$(CONFIG_LIBELF) += genelf.o
 
 CFLAGS_config.o   += -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
 

new file mode 100644
index 000000000000..a1e99da0715a
@@ -0,0 +1,15 @@
+#ifndef __JIT_H__
+#define __JIT_H__
+
+#include <data.h>
+
+extern int jit_process(struct perf_session *session,
+		       struct perf_data_file *output,
+		       struct machine *machine,
+		       char *filename,
+		       pid_t pid,
+		       u64 *nbytes);
+
+extern int jit_inject_record(const char *filename);
+
+#endif /* __JIT_H__ */

2.5.0

See http://www.infradead.org/rpr.html

From: Stephane Eranian <[email protected]>

This is a standalone JVMTI library to help profile Java jitted code with perf record/perf report. The library is not installed or compiled automatically by perf Makefile. It is not used directly by perf. It is arch agnostic and has been tested on X86 and ARM. It needs to be used with a Java runtime, such as OpenJDK, as follows:

  $ java -agentpath:libjvmti.so .......

See the "Committer Notes" below on how to build it.

When used this way, java will generate a jitdump binary file in $HOME/.debug/java/jit/java-jit-*

This binary dump file contains information to help symbolize and annotate jitted code.

The jitdump information must be injected into the perf.data file using:

  $ perf inject --jit -i perf.data -o perf.data.jitted

This injects the MMAP records to cover the jitted code and also generates one ELF image for each jitted function. The ELF images are created in the same subdir as the jitdump file. The MMAP records point there too.

Then, to visualize the function or asm profile, simply use the regular perf commands:

  $ perf report -i perf.data.jitted

or

  $ perf annotate -i perf.data.jitted

JVMTI agent code adapted from the OProfile's opagent code.

This version of the JVMTI agent is using the CLOCK_MONOTONIC as the time source to timestamp jit samples. To correlate with perf_events samples, it needs to run on kernel 4.0.0-rc5+ or later with the following commit from Peter Zijlstra:

34f439278cef ("perf: Add per event clockid support")

With this patch recording jitted code is done as follows:

   $ perf record -k mono -- java -agentpath:libjvmti.so .......

--------------------------------------------------------------------------

Committer Notes:

Extended testing instructions:

  $ cd tools/perf/jvmti/
  $ dnf install java-devel
  $ make

Then, create some simple java stuff to record some samples:

  $ cat hello.java
  public class hello {
	public static void main(String[] args) {
                 System.out.println("Hello, World");
       	}
  }
  $ javac hello.java
  $ java hello
  Hello, World
  $

And then record it using this jvmti thing:

  $ perf record -k mono java -agentpath:/home/acme/git/linux/tools/perf/jvmti/libjvmti.so hello
  java: jvmti: jitdump in /home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jit-1908.dump
  Hello, World
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.030 MB perf.data (268 samples) ]
  $

Now lets insert the PERF_RECORD_MMAP2 records to point jitted mmaps to files created by the agent:

  $ perf inject --jit -i perf.data -o perf.data.jitted

And finally see that it did its job:

  $ perf report -D -i perf.data.jitted | grep PERF_RECORD_MMAP2 | tail -5
  79197149129422 0xfe10 [0xa0]: PERF_RECORD_MMAP2 1908/1923: [0x7f172428bd60(0x80) @ 0x40 fd:02 1840554 1]: --xs /home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jitted-1908-283.so
  79197149235701 0xfeb0 [0xa0]: PERF_RECORD_MMAP2 1908/1923: [0x7f172428ba60(0x180) @ 0x40 fd:02 1840555 1]: --xs /home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jitted-1908-284.so
  79197149250558 0xff50 [0xa0]: PERF_RECORD_MMAP2 1908/1923: [0x7f172428b860(0x180) @ 0x40 fd:02 1840556 1]: --xs /home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jitted-1908-285.so
  79197149714746 0xfff0 [0xa0]: PERF_RECORD_MMAP2 1908/1923: [0x7f172428b660(0x180) @ 0x40 fd:02 1840557 1]: --xs /home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jitted-1908-286.so
  79197149806558 0x10090 [0xa0]: PERF_RECORD_MMAP2 1908/1923: [0x7f172428b460(0x180) @ 0x40 fd:02 1840558 1]: --xs /home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jitted-1908-287.so
  $

So:

  $ perf report -D -i perf.data | grep PERF_RECORD_MMAP2 | wc -l
  Failed to open /tmp/perf-1908.map, continuing without symbols
  21
  $ perf report -D -i perf.data.jitted | grep PERF_RECORD_MMAP2 | wc -l
  307
  $ echo $((307 - 21))
  286
  $

286 extra PERF_RECORD_MMAP2 records.

All for thise tiny, with just one function, ELF files:

  $ file /home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jitted-1908-9.so
  /home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jitted-1908-9.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), corrupted program header size, BuildID[sha1]=ae54a2ebc3ecf0ba547bfc8cabdea1519df5203f, not stripped
  $ readelf -sw /home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jitted-1908-9.so

Symbol table '.symtab' contains 2 entries:

   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 0000000000000040     9 FUNC    LOCAL  DEFAULT    1 atomic_cmpxchg_long

$

Inserted into the build-id cache:

  $ ls -la ~/.debug/.build-id/ae/54a2ebc3ecf0ba547bfc8cabdea1519df5203f
  lrwxrwxrwx. 1 acme acme 111 Feb  5 11:30 /home/acme/.debug/.build-id/ae/54a2ebc3ecf0ba547bfc8cabdea1519df5203f -> ../../home/acme/.debug/jit/java-jit-20160205.XXWIEDls/jitted-1908-9.so/ae54a2ebc3ecf0ba547bfc8cabdea1519df5203f

Note: check why 'file' reports that 'corrupted program header size'.

With a stupid java hog to do some profiling:

$ cat hog.java
  public class hog {
	private static double do_something_else(int i) {
		double total = 0;
		while (i > 0) {
			total += Math.log(i--);
		}
		return total;
	}
	private static double do_something(int i) {
		double total = 0;
		while (i > 0) {
			total += Math.sqrt(i--) + do_something_else(i / 100);
		}
		return total;
	}
	public static void main(String[] args) {
		System.out.println(String.format("%s=%f & %f", args[0],
				   do_something(Integer.parseInt(args[0])),
				   do_something_else(Integer.parseInt(args[1]))));
	}
  }
  $ javac hog.java
  $ perf record -F 10000 -g -k mono java -agentpath:/home/acme/git/linux/tools/perf/jvmti/libjvmti.so hog 100000 2345000
  java: jvmti: jitdump in /home/acme/.debug/jit/java-jit-20160205.XX4sqd14/jit-8670.dump
  100000=291561592.669602 & 32050989.778714
  [ perf record: Woken up 6 times to write data ]
  [ perf record: Captured and wrote 1.536 MB perf.data (12538 samples) ]
  $ perf inject --jit -i perf.data -o perf.data.jitted

Looking at the 'perf report' TUI, at one expanded callchain leading to the jitted code:

  $ perf report --no-children -i perf.data.jitted

Samples: 12K of event 'cycles:pp', Event count (approx.): 3829569932

  Overhead  Comm  Shared Object       Symbol
-   93.38%  java  jitted-8670-291.so  [.] class hog.do_something_else(int)

class hog.do_something_else(int)
- Interpreter
- 75.86% call_stub
JavaCalls::call_helper
jni_invoke_static
jni_CallStaticVoidMethod
JavaMain
start_thread
- 17.52% JavaCalls::call_helper
jni_invoke_static
jni_CallStaticVoidMethod
JavaMain
start_thread

---
 tools/perf/jvmti/Makefile      |  76 +++++++
 tools/perf/jvmti/jvmti_agent.c | 465 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/jvmti/jvmti_agent.h |  29 +++
 tools/perf/jvmti/libjvmti.c    | 208 ++++++++++++++++++
 4 files changed, 778 insertions(+)
 create mode 100644 tools/perf/jvmti/Makefile
 create mode 100644 tools/perf/jvmti/jvmti_agent.c
 create mode 100644 tools/perf/jvmti/jvmti_agent.h
 create mode 100644 tools/perf/jvmti/libjvmti.c

new file mode 100644
index 000000000000..5968f8332a28
@@ -0,0 +1,76 @@
+ARCH=$(shell uname -m)
+
+ifeq ($(ARCH), x86_64)
+JARCH=amd64
+endif
+ifeq ($(ARCH), armv7l)
+JARCH=armhf
+endif
+ifeq ($(ARCH), armv6l)
+JARCH=armhf
+endif
+ifeq ($(ARCH), aarch64)
+JARCH=aarch64
+endif
+ifeq ($(ARCH), ppc64)
+JARCH=powerpc
+endif
+ifeq ($(ARCH), ppc64le)
+JARCH=powerpc
+endif
+
+DESTDIR=/usr/local
+
+VERSION=1
+REVISION=0
+AGE=0
+
+LN=ln -sf
+RM=rm
+
+SLIBJVMTI=libjvmti.so.$(VERSION).$(REVISION).$(AGE)
+VLIBJVMTI=libjvmti.so.$(VERSION)
+SLDFLAGS=-shared -Wl,-soname -Wl,$(VLIBJVMTI)
+SOLIBEXT=so
+
+# The following works at least on fedora 23, you may need the next
+# line for other distros.
+JDIR=$(shell alternatives --display java | tail -1 | cut -d' ' -f 5 | sed 's%/jre/bin/java.%%g')
+#JDIR=$(shell /usr/sbin/update-java-alternatives -l | head -1 | cut -d ' ' -f 3)
+# -lrt required in 32-bit mode for clock_gettime()
+LIBS=-lelf -lrt
+INCDIR=-I $(JDIR)/include -I $(JDIR)/include/linux
+
+TARGETS=$(SLIBJVMTI)
+
+SRCS=libjvmti.c jvmti_agent.c
+OBJS=$(SRCS:.c=.o)
+SOBJS=$(OBJS:.o=.lo)
+OPT=-O2 -g -Werror -Wall
+
+CFLAGS=$(INCDIR) $(OPT)
+
+all: $(TARGETS)
+
+.c.o:
+	$(CC) $(CFLAGS) -c $*.c
+.c.lo:
+	$(CC) -fPIC -DPIC $(CFLAGS) -c $*.c -o $*.lo
+
+$(OBJS) $(SOBJS): Makefile jvmti_agent.h ../util/jitdump.h
+
+$(SLIBJVMTI):  $(SOBJS)
+	$(CC) $(CFLAGS) $(SLDFLAGS)  -o $@ $(SOBJS) $(LIBS)
+	$(LN) $@ libjvmti.$(SOLIBEXT)
+
+clean:
+	$(RM) -f *.o *.so.* *.so *.lo
+
+install:
+	-mkdir -p $(DESTDIR)/lib
+	install -m 755 $(SLIBJVMTI) $(DESTDIR)/lib/
+	(cd $(DESTDIR)/lib; $(LN) $(SLIBJVMTI) $(VLIBJVMTI))
+	(cd $(DESTDIR)/lib; $(LN) $(SLIBJVMTI) libjvmti.$(SOLIBEXT))
+	ldconfig
+
+.SUFFIXES: .c .S .o .lo

new file mode 100644
index 000000000000..8251a1c5ee3f
@@ -0,0 +1,29 @@
+#ifndef __JVMTI_AGENT_H__
+#define __JVMTI_AGENT_H__
+
+#include <sys/types.h>
+#include <stdint.h>
+#include <jvmti.h>
+
+#define __unused __attribute__((unused))
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+void *jvmti_open(void);
+int   jvmti_close(void *agent);
+int   jvmti_write_code(void *agent, char const *symbol_name,
+		       uint64_t vma, void const *code,
+		       const unsigned int code_size);
+int   jvmti_write_debug_info(void *agent,
+		             uint64_t code,
+			     const char *file,
+			     jvmtiAddrLocationMap const *map,
+			     jvmtiLineNumberEntry *tab, jint nr);
+
+#if defined(__cplusplus)
+}
+
+#endif
+#endif /* __JVMTI_H__ */

2.5.0

From: Stephane Eranian <[email protected]>

This patch adds source line information support to perf for jitted code.

The source line info must be emitted by the runtime, such as JVMTI.

Perf injects extract the source line info from the jitdump file and adds the corresponding .debug_lines section in the ELF image generated for each jitted function.

The source line enables matching any address in the profile with a source file and line number.

The improvement is visible in perf annotate with the source code displayed alongside the assembly code.

The dwarf code leverages the support from OProfile which is also released under GPLv2. Copyright 2007 OProfile authors.

---
 tools/perf/jvmti/jvmti_agent.c |  32 +--
 tools/perf/jvmti/jvmti_agent.h |  11 +-
 tools/perf/jvmti/libjvmti.c    | 122 ++++++++-
 tools/perf/util/Build          |   3 +
 tools/perf/util/genelf.c       |  15 +-
 tools/perf/util/genelf.h       |   6 +-
 tools/perf/util/genelf_debug.c | 610 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/jitdump.c      |   8 +-
 8 files changed, 768 insertions(+), 39 deletions(-)
 create mode 100644 tools/perf/util/genelf_debug.c

index cbab139de5a4..6461e02ab940 100644
@@ -374,20 +374,20 @@ jvmti_write_code(void *agent, char const *sym,
 
 int
 jvmti_write_debug_info(void *agent, uint64_t code, const char *file,
-		       jvmtiAddrLocationMap const *map,
-		       jvmtiLineNumberEntry *li, jint num)
+		       jvmti_line_info_t *li, int nr_lines)
 {
-	static const char *prev_str = "\xff";
 	struct jr_code_debug_info rec;
 	size_t sret, len, size, flen;
 	size_t padding_count;
+	uint64_t addr;
+	const char *fn = file;
 	FILE *fp = agent;
 	int i;
 
 	/*
 	 * no entry to write
 	 */
-	if (!num)
+	if (!nr_lines)
 		return 0;
 
 	if (!fp) {

@@ -401,17 +401,18 @@ jvmti_write_debug_info(void *agent, uint64_t code, const char *file,
 	size            = sizeof(rec);
 	rec.p.timestamp = perf_get_timestamp();
 	rec.code_addr   = (uint64_t)(uintptr_t)code;
-	rec.nr_entry    = num;
+	rec.nr_entry    = nr_lines;
 
 	/*
 	 * on disk source line info layout:
 	 * uint64_t : addr
 	 * int      : line number
+	 * int      : column discriminator
 	 * file[]   : source file name
 	 * padding  : pad to multiple of 8 bytes
 	 */
-	size += num * (sizeof(uint64_t) + sizeof(int));
-	size += flen + (num - 1) * 2;
+	size += nr_lines * sizeof(struct debug_entry);
+	size += flen * nr_lines;
 	/*
 	 * pad to 8 bytes
 	 */

@@ -429,28 +430,27 @@ jvmti_write_debug_info(void *agent, uint64_t code, const char *file,
 	if (sret != 1)
 		goto error;
 
-	for (i = 0; i < num; i++) {
-		uint64_t addr;
+	for (i = 0; i < nr_lines; i++) {
 
-		addr = (uint64_t)map[i].start_address;
+		addr = (uint64_t)li[i].pc;
 		len  = sizeof(addr);
 		sret = fwrite_unlocked(&addr, len, 1, fp);
 		if (sret != 1)
 			goto error;
 
-		len  = sizeof(int);
+		len  = sizeof(li[0].line_number);
 		sret = fwrite_unlocked(&li[i].line_number, len, 1, fp);
 		if (sret != 1)
 			goto error;
 
-		if (i == 0) {
-			sret = fwrite_unlocked(file, flen, 1, fp);
-		} else {
-			sret = fwrite_unlocked(prev_str, 2, 1, fp);
-		}
+		len  = sizeof(li[0].discrim);
+		sret = fwrite_unlocked(&li[i].discrim, len, 1, fp);
 		if (sret != 1)
 			goto error;
 
+		sret = fwrite_unlocked(fn, flen, 1, fp);
+		if (sret != 1)
+			goto error;
 	}
 	if (padding_count)
 		sret = fwrite_unlocked(pad_bytes, padding_count, 1, fp);

index 8251a1c5ee3f..bedf5d0ba9ff 100644
@@ -11,16 +11,23 @@
 extern "C" {
 #endif
 
+typedef struct {
+	unsigned long	pc;
+	int		line_number;
+	int		discrim; /* discriminator -- 0 for now */
+} jvmti_line_info_t;
+
 void *jvmti_open(void);
 int   jvmti_close(void *agent);
 int   jvmti_write_code(void *agent, char const *symbol_name,
 		       uint64_t vma, void const *code,
 		       const unsigned int code_size);
+
 int   jvmti_write_debug_info(void *agent,
 		             uint64_t code,
 			     const char *file,
-			     jvmtiAddrLocationMap const *map,
-			     jvmtiLineNumberEntry *tab, jint nr);
+			     jvmti_line_info_t *li,
+			     int nr_lines);
 
 #if defined(__cplusplus)
 }

index 52a4a806ee2f..a34752d28488 100644
@@ -108,8 +108,11 @@ libperf-$(CONFIG_LZMA) += lzma.o
 libperf-y += demangle-java.o
 libperf-$(CONFIG_LIBELF) += jitdump.o
 libperf-$(CONFIG_LIBELF) += genelf.o
+libperf-$(CONFIG_LIBELF) += genelf_debug.o
 
 CFLAGS_config.o   += -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
+# avoid compiler warnings in 32-bit mode
+CFLAGS_genelf_debug.o  += -Wno-packed
 
 $(OUTPUT)util/parse-events-flex.c: util/parse-events.l $(OUTPUT)util/parse-events-bison.c
 	$(call rule_mkdir)

index 145f8116ef56..c1ef805c6a8f 100644
@@ -156,7 +156,8 @@ gen_build_id(struct buildid_note *note, unsigned long load_addr, const void *cod
  */
 int
 jit_write_elf(int fd, uint64_t load_addr, const char *sym,
-	      const void *code, int csize)
+	      const void *code, int csize,
+	      void *debug, int nr_debug_entries)
 {
 	Elf *e;
 	Elf_Data *d;

@@ -385,9 +386,15 @@ jit_write_elf(int fd, uint64_t load_addr, const char *sym,
 	shdr->sh_size = sizeof(bnote);
 	shdr->sh_entsize = 0;
 
-	if (elf_update(e, ELF_C_WRITE) < 0) {
-		warnx("elf_update 4 failed");
-		goto error;
+	if (debug && nr_debug_entries) {
+		retval = jit_add_debug_info(e, load_addr, debug, nr_debug_entries);
+		if (retval)
+			goto error;
+	} else {
+		if (elf_update(e, ELF_C_WRITE) < 0) {
+			warnx("elf_update 4 failed");
+			goto error;
+		}
 	}
 
 	retval = 0;

index d8e9ece13c8b..45bf9c6d3257 100644
@@ -3,7 +3,11 @@
 
 /* genelf.c */
 extern int jit_write_elf(int fd, uint64_t code_addr, const char *sym,
-			 const void *code, int csize);
+			 const void *code, int csize,
+			 void *debug, int nr_debug_entries);
+/* genelf_debug.c */
+extern int jit_add_debug_info(Elf *e, uint64_t code_addr,
+			      void *debug, int nr_debug_entries);
 
 #if   defined(__arm__)
 #define GEN_ELF_ARCH	EM_ARM

index 9f7a01289efe..99fa5eee9fe0 100644
@@ -63,7 +63,9 @@ jit_emit_elf(char *filename,
 	     const char *sym,
 	     uint64_t code_addr,
 	     const void *code,
-	     int csize)
+	     int csize,
+	     void *debug,
+	     int nr_debug_entries)
 {
 	int ret, fd;
 

@@ -76,7 +78,7 @@ jit_emit_elf(char *filename,
 		return -1;
 	}
 
-        ret = jit_write_elf(fd, code_addr, sym, (const void *)code, csize);
+        ret = jit_write_elf(fd, code_addr, sym, (const void *)code, csize, debug, nr_debug_entries);
 
         close(fd);
 

@@ -347,7 +349,7 @@ static int jit_repipe_code_load(struct jit_buf_desc *jd, union jr_entry *jr)
 
 	size = PERF_ALIGN(size, sizeof(u64));
 	uaddr = (uintptr_t)code;
-	ret = jit_emit_elf(filename, sym, addr, (const void *)uaddr, csize);
+	ret = jit_emit_elf(filename, sym, addr, (const void *)uaddr, csize, jd->debug_data, jd->nr_debug_entries);
 
 	if (jd->debug_data && jd->nr_debug_entries) {
 		free(jd->debug_data);

2.5.0

Pulled, thanks a lot Arnaldo!

Ingo